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Abstract 


The  majority  of  the  Air  Force’s  stateside  utility  systems  are  old,  obsolete,  and 
unreliable.  The  cost  to  upgrade  and  repair  these  systems  is  currently  estimated  at  over  $4 
billion.  In  response,  the  Air  Force  began  efforts  to  convey  ownership  of  these  utility 
systems  to  the  private  sector  through  privatization  efforts.  However,  privatization  critics 
believe  that  newly  privatized  entities  of  government  will  cost  the  public  more  money  and 
provide  a  lower  level  of  service.  Therefore,  independent  audits  are  a  necessity  to  ensure 
government  initiatives,  meet  their  intended  goals.  However,  the  Air  Force  currently  lacks 
an  effective  auditing  tool  to  ensure  the  efficiency  and  lower  cost  associated  with  utility 
privatization  are  balanced  with  the  desired  increases  in  quality,  reliability,  and 
responsiveness  of  its  utility  systems. 

The  Value-Focused  Thinking  methodology  was  used  to  create  a  multi-objective 
decision  analysis  model  to  determine  the  effectiveness  of  Air  Force  utility  privatization 
efforts  by  evaluating  the  performance  of  privatized  utility  systems.  Consisting  of  28 
bottom-tier  values  and  47  measures,  the  model  captures  the  majority  of  the  Air  Force’s 
objectives  and  concerns  regarding  its  privatized  utility  systems.  Using  notational  data, 
the  utility  systems  at  eight  simulated  Air  Force  installations  were  evaluated  and  rank 
ordered  to  validate  the  model.  Sensitivity  analysis  was  also  performed  to  provide  further 
insight  into  the  decision  making  process.  The  results  of  this  research  prove  that  the 
model  can  be  an  effective  decision  analysis  tool  that  provides  the  Air  Force  insight  on  the 
performance  of  its  privatized  utility  systems. 


IV 


Acknowledgments 


I  would  like  to  acknowledge  the  help  of  many  people  during  this  research.  First,  I 
would  like  to  thank  the  members  of  my  thesis  committee.  I  thank  Lieutenant  Colonel 
Thai,  my  advisor,  for  his  direction,  support,  and  patience  during  my  research.  Thanks  to 
Major  Weir  for  his  selfless  giving  of  his  time  when  he  had  so  many  other  things  going  on 
and  for  showing  me  all  of  those  “cool  tricks”  in  Logical  Decisions.  Thanks  also  go  to 
Captain  Sayeed  for  his  help  with  creating  the  evaluation  model  and  general  advice. 

Thanks  are  due  to  the  individuals  whom  have  mentored  and  helped  me  throughout 
my  career.  I  would  particularly  like  to  thank  Brigadier  General  Eulberg  for  his  belief  in 
me  as  well  as  getting  me  back  into  CE.  To  my  former  supervisors,  Colonel  Patrick, 
Lieutenant  Colonel  Jackson,  Bruce  Nadler,  Leon  Spradling,  and  Timothy  Hendrix,  thanks 
for  providing  me  with  the  opportunity  to  learn  and  succeed.  Thanks  also  go  to  Roger 
Kiker  for  teaching  me  the  utility  system  management  business. 

I  would  also  like  to  thank  my  classmates  in  the  GEM  program,  their  insights  and 
fellowship  made  my  time  at  AFIT  bearable.  I  would  especially  like  to  thank  Captain 
“Heath”  Duncan  for  all  of  his  assistance  during  the  research  and  for  being  a  good  friend. 

Most  importantly,  I  want  to  say  thank  you  to  my  family  for  all  of  their  love  and 
sacrifice.  Special  thanks  go  to  my  lovely  wife  for  her  understanding,  endless  patience 
and  encouragement  when  it  was  most  required. 


Captain  Carlos  Braziel 


v 


Table  of  Contents 


Page 

Abstract . iv 

Acknowledgments . v 

Table  of  Contents . vi 

List  of  Figures . xi 

List  of  Tables . xiv 

Chapter  1.  Introduction . 1 

1 . 1  General  Background . 1 

1 .2  Specific  Background . 1 

1 .3  Problem  Statement . 3 

1.4  Research  Objective  and  Investigative  Questions . 4 

1 .5  Methodology . 5 

1 .6  Research  Scope  and  Limitations . 5 

1.7  Document  Structure . 6 

Chapter  2.  Background  of  the  Problem/Literature  Review . 7 

2.1  Privatization  in  General . 7 

2.1.1  Privatization  Defined . 8 

2.1.2  History  of  Privatization . 9 

2.2  Utility  Privatization . 10 

2.2.1  Utility  System  and  Utility  Privatization  Defined . 1 1 

2.2.2  History  of  Air  Force  Utility  Privatization . 12 

2.2.3  Air  Force  Utility  Privatization  Laws  and  Directives . 12 

2.2.4  Contracting  Process . 14 

2.2.4. 1  Phase  I  -  Feasibility  Analysis  and  Market  Review . 15 

2.2. 4.2  Phase  II  -  Comprehensive  Analysis . 15 

2.2. 4. 3  Phase  III  -  Implementation . 16 

2.3  Privatization  Debate . 17 

2.4  Previous  Research . 20 

2.5  Decision  Analysis . 23 

2.5.1  Introduction  to  Decision  Analysis . 23 

2.5.2  Alternative-Focused  Thinking  versus  Value-Focused  Thinking . 24 

2.6  Value-Focused  Thinking . 25 


vi 


2.7  Ten-Step  Value-Focused  Thinking  Process . 27 

2.7.1  Step  1  -  Problem  Identification . 28 

2.7.2  Step  2  -  Value  Hierarchy  Construction . 29 

2.7.2. 1  Desired  Properties  of  Value  Hierarchies . 29 

2.1 22  Generation  of  Values . 31 

2.1 22  Structuring  the  Value . 33 

2.7.3  Step  3  -  Evaluation  Measure  Development . 34 

2.7.3. 1  Types  of  Evaluation  Measure  Scales . 35 

2.12.2  Desired  Properties  of  Value  Hierarchies . 36 

2.7.4  Step  4  -  Value  Function  Creation . 36 

2.7.5  Step  5  -  Value  Hierarchy  Weighting . 38 

2.7.5. 1  Local  Weighting . 39 

2.1.52  Global  Weighting . 40 

2.7.6  Step  6  -  Alternative  Generation . 41 

2.7.7  Step  7  -  Alternative  Scoring . 42 

2.7.8  Step  8  -  Deterministic  Analysis . 42 

2.7.9  Step  9  -  Conduct  Sensitivity  Analysis . 43 

2.7.10  Step  10  -  Recommendations  and  Conclusions . 43 

Chapter  3.  Methodology . 45 

3.1  Step  1  -  Problem  Identification . 45 

3.2  Step  2  -  Value  Hierarchy  Construction . 46 

3.2.1  Decompose  Fundamental  Objective . 50 

3.2.2  Quality  Branch . 50 

3.2.2. 1  Effective  Administration . 53 

3.2. 2. 1.1  Maintaining  Proper  Licenses,  Permits,  &  Certifications . 53 

3.2. 2. 1.2  Maintaining  Service  Records  for  Two  Years . 53 

3.2. 2. 1.3  Maintaining  and  Updating  Drawings . 54 

3.2. 2.2  Environmental  Stewardship . 54 

3.2. 2.2.1  Effective  Spill  Contingency  Plan . 55 

3.2. 2.2. 2  Hazardous  Material/Waste  Minimization  and  Recycling . 55 

3.2. 2. 3  Utility  System  Safety . 55 

3.2. 2. 3.1  Compliance  with  Utility  System  Laws/Regulations . 56 

3.2. 2. 3. 2  Decreased  Utility  System  Mishaps . 56 

3. 2. 2. 3. 3  Employee  Safety  Certification . 56 

vii 


3.2. 2.4  Utility  System  Security . 56 

3.2. 2. 5  Sub-Metering  Capability . 57 

3.2.3  Reliability  Branch . 57 

3.2.3. 1  Completed  Renewals/Replacements . 58 

3.2. 3.2  Decreased  Utility  System  Outages . 58 

3.2.4  Responsiveness  Branch . 58 

3.2.4. 1  Effective  Digging  Permits/Line  Marking  Program . 59 

3.2. 4.2  High  Contracting  Meeting  Attendance . 61 

3. 2. 4. 3  Timely  Meter  Readings . 61 

3. 2. 4.4  Timely  Service . 61 

3.2. 4.4.1  Timely  Emergency  Service  Response . 61 

3.2. 4.4. 2  Timely  Urgent  Service  Response . 63 

3.2. 4.4. 3  Timely  Routine  Service  Response . 63 

3.3  Step  3  -  Develop  Evaluation  Measures . 65 

3.4  Step  4  -  Create  Value  Functions . 67 

3.5  Step  5  -  Weight  the  Value  Hierarchy . 71 

3.5.1  Assignment  of  Local  Weights . 72 

3. 5. 1.1  Local  Weights  for  Quality  Branch . 73 

3.5. 1.2  Local  Weights  fox  Reliability  Branch . 76 

3.5. 1.3  Local  Weights  fox  Responsiveness  Branch . 76 

3.5.2  Assignment  of  Global  Weights . 79 

3.6  Summary . 80 

Chapter  4.  Results  and  Analysis . 81 

4. 1  Step  6  -  Alternative  Generation . 81 

4.2  Step  7  -  Alternative  Scoring . 82 

4.3  Step  8  -  Deterministic  Analysis . 83 

4.4  Step  9  -  Sensitivity  Analysis . 84 

4.4.1  Quality  Branch  Sensitivity  Analysis . 85 

4.4.2  Reliability  Sensitivity  Analysis . 87 

4.4.3  Responsiveness  Sensitivity  Analysis . 90 

Chapter  5.  Findings  and  Conclusions . 93 

5.1  Review . 93 

5.2  Model  Strengths . 95 

5.3  Model  Limitations . 95 


viii 


5.4  Conclusions . 96 

5.5  Recommendations  for  Future  Work . 96 

Appendix  A.  Value  Hierarchy  Measures . 97 

Appendix  B.  Value  Hierarchy  Definitions . 104 

Appendix  C.  Single  Dimension  Value  Functions  (SDVFs) . Ill 

SDVF  #  1 — Percentage  of  Up-to-Date  Licenses,  Permits,  and  Certifications . Ill 

SDVF  #  2 — Are  Records  for  the  Past  2  Years  Maintained  Properly . 112 

SDVF  #  3 — Average  Number  of  Days  to  Update  Drawings . 113 

SDVF  #  4 — Number  of  Positive  Findings  for  Spill  Contingency  Plan . 1 14 

SDVF  #  5 — Number  of  Minor  Findings  for  Spill  Contingency  Plan . 115 

SDVF  #  6 — Number  of  Major  Findings  for  Spill  Contingency  Plan . 1 16 

SDVF  #  7 — Number  of  Significant  Findings  for  Spill  Contingency  Plan . 117 

SDVF  #  8 — Percentage  of  Liquid  Waste  Diverted  from  Landfills . 118 

SDVF  #  9 — Percentage  of  Solid  Waste  Diverted  from  Landfills . 119 

SDVF  #  10 — Number  of  Positive  Findings  for  Recycling . 120 

SDVF  #  1 1 — Number  of  Minor  Findings  for  Recycling . 121 

SDVF  #  12 — Number  of  Major  Findings  for  Recycling . 122 

SDVF  #  13 — Number  of  Significant  Findings  for  Recycling . 123 

SDVF  #  14 — Number  of  Lost  Man-Hours  Due  to  Utility  System  Mishaps . 124 

SDVF  #  15 — Number  of  Utility  System  Mishaps . 125 

SDVF  #  16 — Number  of  RAC  1  -  Catastrophic  Violations . 126 

SDVF  #  17 — Number  of  RAC  2  -  Critical  Violations . 127 

SDVF  #  18 — Number  of  RAC  3  -  Moderate  Violations . 128 

SDVF  #  19 — Number  of  RAC  4  -  Negligible  Violations . 129 

SDVF  #  20 — Percentage  of  Employees  Completing  all  Requirements . 130 

SDVF  #  21 — Percentage  of  Meters  Calibrated  from  Random  Sample . 131 

SDVF  #  22 — Percentage  of  Total  Facilities  Metered . 132 

SDVF  #  23 — Number  of  Employees  Identified  as  Potential  Threats . 133 

SDVF  #  24 — Are  all  Employee  Security  Clearances  Up-to-Date . 134 

SDVF  #  25 — Percentage  of  Items  Actually  Replaced . 135 

SDVF  #  26 — Percentage  of  Critical  Outages  Caused  by  System  Management . 136 

SDVF  #  27 — Percentage  of  Non-Critical  Outages  Caused  by  System  Management  137 

SDVF  #  28 — Number  of  Utility  Line  Hits . 138 

SDVF  #  29 — Percentage  of  Meetings  Attended . 139 

ix 


SDVF  #  30 — Percentage  of  Late  Meter  Readings . 140 

SDVF  #  31 — Is  There  an  Adequate  24/7  Hotline . 141 

SDVF  #  32 — Percentage  of  Goal  Met  for  Timely  Initial  Emergency  Response . 142 

SDVF  #  33 — Percentage  of  Goal  Met  for  Timely  Emergency  Crew  Response . 143 

SDVF  #  34 — Percentage  of  Goal  Met  for  Timely  Remedied  Emergency  Response.  144 

SDVF  #  35 — Number  of  Outstanding  Ratings  for  Exercises/Contingencies . 145 

SDVF  #  36 — Number  of  Excellent  Ratings  from  Exercises/Contingencies . 146 

SDVF  #  37 — Number  of  Satisfactory  Ratings  from  Exercises/Contingencies . 147 

SDVF  #  38 — Number  of  Marginal  Ratings  from  Exercises/Contingencies . 148 

SDVF  #  39 — Number  of  Unsatisfactory  Ratings  from  Exercises/Contingencies . 149 

SDVF  #  40 — Percentage  of  Goal  Met  for  Effective  FACMAN 

Emergency  Coordination . 150 

SDVF  #  41 — Percentage  of  Goal  Met  for  Timely  Initial  Urgent  Response . 151 

SDVF  #  42 — Percentage  of  Goal  Met  for  Timely  Remedied  Urgent  Response . 152 

SDVF  #  43 — Percentage  of  Goal  Met  for  Effective  FACMAN 

Urgent  Coordination . 153 

SDVF  #  44 — Percentage  of  Goal  Met  for  Timely  Initial  Routine  Response . 154 

SDVF  #  45 — Percentage  of  Goal  Met  for  Timely  Remedied  Routine  Response 155 

SDVF  #  46 — Percentage  of  Goal  Met  for  Effective  2-Week  Coordination . 156 

SDVF  #  47 — Percentage  of  Goal  Met  for  Effective  FACMAN 

Routine  Coordination . 157 

Appendix  D.  Global  Weights  for  Utility  Privatization  Evaluation  Hierarchy . 158 

Appendix  E.  Ranked  Ordered  Global  Weights . 166 

Appendix  F.  Notional  Data  Scoring  Results . 168 

Bibliography . 174 

Vita . 177 


x 


List  of  Figures 


Page 

Figure  1.  Alternative  and  Value-Focused  Thinking  Approaches  . 24 

Figure  2.  Benefits  of  Value-Focused  Thinking . 27 

Figure  3.  VFT:  Ten-Step  Process . 28 

Figure  4.  “Buy  the  Best  Truck”  Hierarchy  . 34 

Figure  5.  Monotonically  Increasing  Exponential  (left)  and  Piecewise  Linear  (right) 

Value  Functions . 37 

Figure  6.  Monotonically  Decreasing  Exponential  (left)  and  Piecewise  Linear  (right) 

Value  Functions . 38 

Figure  7.  “Buy  the  Best  Truck”  Value  Hierarchy  with  Local  Weights  . 39 

Figure  8.  “Buy  the  Best  Truck”  Local  Weights  Example . 40 

Figure  9.  “Buy  the  Best  Truck”  Global  Weights  Example . 41 

Figure  10.  Strawman  Hierarchy . 47 

Figure  1 1 .  Final  Utility  Privatization  Evaluation  Value  Hierarchy . 49 

Figure  12.  Utility  Privatization  Evaluation  Hierarchy  Showing  Tiers  1  and  2 . 51 

Figure  13.  Utility  Privatization  Evaluation  Hierarchy  for  Quality  Branch . 52 

Figure  14.  Utility  Privatization  Evaluation  Hierarchy  for  Reliability  Branch . 57 

Figure  15.  Utility  Privatization  Evaluation  Hierarchy  for  Responsiveness  Branch . 60 

Figure  16.  Average  Number  of  Days  to  Update  SDVF . 69 

Figure  17.  Percentage  of  Up-to-Date  Licenses,  Permits,  Permits,  and 

Certifications  SDVF . 70 

Figure  18.  Number  of  Minor  Findings  for  Spill  Contingency  Plan  SDVF . 71 

Figure  19.  Local  Weights  for  the  First-Tier  Values . 72 

Figure  20.  Local  Weights  for  Quality  Branch . 74 

Figure  21.  Local  Weights  for  Quality  Branch  (Continued) . 75 

Figure  22.  Local  Weights  for  Reliability  Branch . 76 

Figure  23.  Local  Weights  for  Responsiveness  Branch . 78 

Figure  24.  Local  Weights  for  Responsiveness  Branch  (Continued) . 79 

Figure  25.  Global  Weight  Calculation  Example . 80 

Figure  26.  Alternative  Score  Ranking  and  Deterministic  Analysis  Results . 83 

Figure  27.  Quality  Branch  Sensitivity  Analysis . 86 

Figure  28.  Reliability  Branch  Sensitivity  Analysis . 88 

Figure  29.  Responsiveness  Branch  Sensitivity  Analysis . 91 


xi 


Figure  30.  Percentage  of  Up-to-Date  Licenses,  Permits,  and  Certifications  SDVF . Ill 

Figure  3 1 .  Are  Records  for  the  Past  2  Years  Maintained  Properly  SDVF . 112 

Figure  32.  Average  Number  of  Days  to  Update  Drawings  SDVF . 113 

Figure  33.  Number  of  Positive  Findings  for  Spill  Contingency  Plan  SDVF . 1 14 

Figure  34.  Number  of  Minor  Findings  for  Spill  Contingency  Plan  SDVF . 115 

Figure  35.  Number  of  Major  Findings  for  Spill  Contingency  Plan  SDVF . 1 16 

Figure  36.  Number  of  Significant  Findings  for  Spill  Contingency  Plan  SDVF . 117 

Figure  37.  Percentage  of  Liquid  Waste  Diverted  from  Landfills  SDVF . 118 

Figure  38.  Percentage  of  Solid  Waste  Diverted  from  Landfills  SDVF . 119 

Figure  39.  Number  of  Positive  Findings  for  Recycling  SDVF . 120 

Figure  40.  Number  of  Minor  Findings  for  Recycling  SDVF . 121 

Figure  41.  Number  of  Major  Findings  for  Recycling  SDVF . 122 

Figure  42.  Number  of  Significant  Findings  for  Recycling  SDVF . 123 

Figure  43.  Number  of  Lost  Man-Hours  Due  to  Utility  System  Mishaps  SDVF . 124 

Figure  44.  Number  of  Utility  System  Mishaps  SDVF . 125 

Figure  45.  Number  of  RAC  1  -  Catastrophic  Violations  SDVF . 126 

Figure  46.  Number  of  RAC  2  -  Critical  Violations  SDVF . 127 

Figure  47.  Number  of  RAC  3  -  Moderate  Violations  SDVF . 128 

Figure  48.  Number  of  RAC  4  -  Negligible  Violations  SDVF . 129 

Figure  49.  Percentage  of  Employees  Completing  all  Requirements  SDVF . 130 

Figure  50.  Percentage  of  Meters  Calibrated  from  Random  Sample  SDVF . 131 

Figure  51.  Percentage  of  Total  Facilities  Metered  SDVF . 132 

Figure  52.  Number  of  Employees  Identified  as  Potential  Threats  SDVF . 133 

Figure  53.  Are  all  Employee  Security  Clearances  Up-to-Date  SDVF . 134 

Figure  54.  Percentage  of  Items  Actually  Replaced  SDVF . 135 

Figure  55.  Percentage  of  Critical  Outages  Caused  by  System  Management  SDVF . 136 

Figure  56.  Percentage  of  Non-Critical  Outages  Caused  by  System 

Management  SDVF . 137 

Figure  57.  Number  of  Utility  Line  Hits  SDVF . 138 

Figure  58.  Percentage  of  Meetings  Attended  SDVF . 139 

Figure  59.  Percentage  of  Late  Meter  Readings  SDVF . 140 

Figure  60.  Is  There  an  Adequate  24/7  Hotline  SDVF . 141 

Figure  61.  Percentage  of  Goal  Met  for  Timely  Initial  Emergency  Response  SDVF  ....  142 

Figure  62.  Percentage  of  Goal  Met  for  Timely  Emergency  Crew  Response  SDVF . 143 

xii 


Figure  63.  Percentage  of  Goal  Met  for  Timely  Remedied  Emergency 

Response  SDVF . 144 

Figure  64.  Number  of  Outstanding  Ratings  for  Exercises/Contingencies  SDVF . 145 

Figure  65.  Number  of  Excellent  Ratings  from  Exercises/Contingencies  SDVF . 146 

Figure  66.  Number  of  Excellent  Ratings  from  Exercises/Contingencies  SDVF . 147 

Figure  67.  Number  of  Marginal  Ratings  from  Exercises/Contingencies  SDVF . 148 

Figure  68.  Number  of  Unsatisfactory  Ratings  from  Exercises/Contingencies  SDVF...  149 

Figure  69.  Percentage  of  Goal  Met  for  Effective  FACMAN  Emergency 

Coordination  SDVF . 150 

Figure  70.  Percentage  of  Goal  Met  for  Timely  Initial  Urgent  Response  SDVF . 151 

Figure  71.  Percentage  of  Goal  Met  for  Timely  Remedied  Urgent  Response  SDVF . 152 

Figure  72.  Percentage  of  Goal  Met  for  Effective  FACMAN  Urgent 

Coordination  SDVF . 153 

Figure  73.  Percentage  of  Goal  Met  for  Timely  Initial  Routine  Response  SDVF . 154 

Figure  74.  Percentage  of  Goal  Met  for  Timely  Remedied  Routine  Response  SDVF  ...  155 

Figure  75.  Percentage  of  Goal  Met  for  Effective  2-Week  Coordination  SDVF . 156 

Figure  76.  Percentage  of  Goal  Met  for  Effective  FACMAN  Routine 

Coordination  SDVF . 157 


xiii 


List  of  Tables 


Page 

Table  1.  Examples  of  Measures . 66 

Table  2.  Example  of  Definitions . 67 

Table  3.  Scoring  Results  for  Alternatives . 82 

Table  4.  Slope  Calculations  for  Quality  Value . 87 

Table  5.  Slope  Calculations  fox  Reliability  Value . 89 

Table  6.  Slope  Calculations  for  Responsiveness  Value . 92 

Table  7.  Summary  of  Measures  for  the  Quality  Branch . 97 

Table  8.  Summary  of  Measures  for  the  Reliability  Branch . 100 

Table  9.  Summary  of  Measures  for  the  Responsiveness  Branch . 100 

Table  10.  Definitions  of  Quality  Measures . 104 

Table  11.  Definitions  of  Reliability  Measures . 107 

Table  12.  Definitions  of  Responsiveness  Measures . 107 


xiv 


Chapter  1.  Introduction 


1.1  General  Background 

With  an  increased  emphasis  on  decreasing  bureaucracy  and  improving  efficiency, 
world  governments  are  using  or  seriously  considering  privatization  as  solution  to  both 
decrease  the  size  of  the  government  and  provide  better  efficiency.  Privatization  has  been 
broadly  regarded  as  .  .the  act  of  reducing  the  role  of  government,  or  increasing  the  role 
of  the  private  sector,  in  an  activity  or  in  the  ownership  of  assets...”  (Savas,  1987:3).  This 
management  concept  is  rapidly  being  incorporated  into  many  local  and  national 
government  operations  across  the  globe.  For  example,  the  World  Bank  has  financed  the 
privatization  ventures  of  over  8,500  state-owned  enterprises  in  over  200  countries  since 
the  early  1990s  (Kikeri  et  al,  1992). 

Today,  almost  every  country  has  a  privatization  program  or  at  least  a  sector  of 
activity  susceptible  to  private  management  if  not  ownership.  For  instance,  the  countries 
of  Bolivia,  Great  Britain,  and  Malaysia  have  privatized  their  water  system,  electrical 
system,  and  National  Lottery  program,  respectively.  These  countries,  along  with  many 
others,  believe  that  switching  publicly  owned  enterprises  to  privately  owned  firms  will 
lead  to  greater  economic  efficiency,  reduced  public  debt,  and  improved  budgetary 
management  (Hartley  et  al,  1991;  Letwin,  1988). 

1.2  Specific  Background 

In  the  United  States,  many  states  and  cities  have  incorporated  privatization  into 
their  public  operations  as  well.  Some  states  are  privatizing  utilities,  prisons,  child  foster 
care,  and  numerous  other  items.  Similarly,  major  cities  in  the  United  States  are  using 


1 


privatization  strategies  for  a  discemable  share  of  municipal  services  (Dilger  et  al,  1997). 
However,  there  has  been  little  progress  at  the  federal  level.  The  change  in  political 
climate  in  the  late  1980s  and  early  1990s,  namely  the  fall  of  the  Soviet  Union  and  the 
American  voters’  clear  discontent  with  an  oversized  government,  has  forced  the  federal 
government  to  take  a  more  active  role  in  privatization  (Savas  2000:38).  Inevitably,  the 
once  plentiful  Department  of  Defense  budget  established  by  the  Reagan  administration 
was  shrunk  tremendously.  Thus,  a  situation  was  created  in  which  the  Air  Force  and  other 
military  services  had  to  make  enonnous  program  and  manning  cuts  in  order  to 
compensate  for  the  smaller  Department  of  Defense  budget. 

As  the  numbers  of  trained  Air  Force  utility  system  technicians  were  voluntarily  or 
involuntarily  lowered,  the  aging  infrastructure  at  most  Air  Force  installations  placed 
increased  demands  on  the  ever-shrinking  operations  and  maintenance  budget.  With  the 
lack  of  sufficient  infrastructure  dollars  and  trained  technicians  to  maintain  them,  the 
utility  systems  at  many  Air  Force  installations  quickly  became  liabilities  instead  of 
reliable  entities  used  to  support  the  mission.  As  a  result,  the  Air  Force  realized  it  must 
find  a  better  and  inexpensive  way  to  provide  installations  with  quality  utility  service 
without  sacrificing  mission  support.  The  Department  of  Defense  and  the  Air  Force 
looked  to  utility  privatization  as  the  way  to  provide  military  installations  with  lower  life- 
cycle  costs  while  increasing  the  reliability  of  utility  service  through  modernization.  In 
the  late  1900s,  the  Air  Force  initiated  the  process  to  privatize  its  electric,  gas,  and  water 
utilities  at  various  stateside  Air  Force  installations.  The  Air  Force’s  objective  in  this 
process  was,  and  continues  to  be,  to  divest  itself  of  those  utility  systems  that  do  not 
directly  support  the  war-fighting  mission  (Department  of  Defense,  2002).  On  those  Air 
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Force  installations  with  utility  systems  that  are  not  considered  a  core  operational 
requirement,  the  Air  Force  plans  to  turn  over  the  business  of  operation  and  maintenance 
to  the  private  sector  for  better  efficiency  (Department  of  Defense,  2002). 

Even  though  the  federal  and  local  state  governments  view  privatization  as  a 
means  to  a  more  efficient  and  less  costly  provision  of  government  services,  there  is 
considerable  opposition  to  its  use.  In  fact,  there  is  criticism  in  the  public  sector  about  the 
way  privatization  decisions  have  been  made  and  how  cost  and  perfonnance 
measurements  were  created.  This  criticism  has  contributed  to  the  belief  that  newly 
privatized  entities  of  government  will  cost  the  public  more  money  and  provide  a  lower 
level  of  service.  Furthermore,  skeptics  of  privatization  caution  that  it  is  important  to  treat 
privatization,  as  with  any  new  government  policy,  as  an  experiment  (Wallin,  1997:19). 

1.3  Problem  Statement 

Independent  audits  on  federal  and  state  policies,  such  as  utility  privatization,  are 
instrumental  in  helping  to  avoid  potential  bias,  limit  errors,  and  alleviate  the  fears  of 
critics.  To  date,  there  does  not  appear  to  be  a  strong  body  of  knowledge  in  the  area  of 
utility  privatization  evaluation,  thus  creating  a  research  gap.  This  research  gap  can  be 
filled  by  combining  the  lessons  learned  from  previous  privatization  initiatives  with  sound 
quantitative/qualitative  measurement  techniques.  The  combination  of  these  two 
dimensions  is  essential  to  the  creation  of  an  effective  audit  tool  for  utility  privatization. 
Careful  oversight  and  monitoring  of  the  Air  Force  utility  privatization  process  will  help 
ensure  that  the  efficiency  and  lower  cost  associated  with  private  business  is  balanced  with 
the  desired  increases  in  quality,  reliability,  and  responsiveness  of  its  utility  systems. 
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Therefore,  the  purpose  of  this  thesis  is  to  develop  a  mathematical  model  to  audit 
utility  privatization  by  means  of  examining  and  comparing  specific  characteristics  or 
values  of  Air  Force  utility  systems.  This  model  must  be  capable  of  facilitating  the 
evaluation  of  utility  privatization  by  providing  a  clear  structure  for  the  multiple  objectives 
associated  with  utility  privatization.  In  addition,  this  model  must  be  capable  of  not  only 
capturing  the  values  (what  is  important  to  the  Air  Force)  of  a  utility  system  but  be  able  to 
use  the  values  to  evaluate  the  effectiveness  of  Air  Force  utility  systems.  Furthermore,  the 
model  must  be  capable  of  balancing  the  “hard”  quantitative  and  “soft”  qualitative  factors 
of  utility  privatization.  Finally,  the  model  must  be  reliable,  repeatable,  and  defendable. 

1.4  Research  Objective  and  Investigative  Questions 

The  Department  of  Defense  and  the  Air  Force  have  invested  a  considerable  amount 
of  resources  on  studies  and  contracting  actions  to  make  utility  privatization  happen. 
Examining,  comparing,  and  identifying  the  values  comprising  the  operation  of  privatized 
and  non-privatized  utility  systems  at  Air  Force  installations  will  provide  insight  into  the 
important  factors  impacting  the  utility  privatization  process.  Therefore,  the  objective  of 
this  thesis  is  to  create  a  valid  mathematical  model  to  determine  the  effectiveness  of  Air 
Force  utility  privatization  by  evaluating  the  performance  of  privatized  Air  Force  utility 
systems.  To  do  this,  the  following  investigative  questions  will  be  addressed  during  this 
research. 

1 .  Given  that  quantitative  data  (performance)  will  be  collected,  what  is  a 
suitable  method  to  evaluate  and  measure  the  overall  effectiveness  of  a 
utility  system?  In  order  to  accomplish  this,  the  “hard”  quantitative  and 
the  “soft”  qualitative  factors  of  utility  privatization  must  be  balanced. 

2.  What  are  the  major  factors  and  sub-factors  that  should  be  considered 
when  evaluating  the  performance  of  Air  Force  utility  privatization? 
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3. 


How  do  the  major  factors  and  sub-factors  compare  to  each  other  in 
terms  of  importance? 

1.5  Methodology 

Value-focused  thinking  (VFT),  as  developed  by  Keeney  (1992),  is  a  modeling 
technique  that  has  been  frequently  used  to  assist  organizations  with  making  decisions. 

The  civilian  sector  has  successfully  applied  VFT  in  the  automotive,  oil,  and 
phannaceutical  industries.  In  particular,  VFT  has  been  very  instrumental  in  structuring 
the  critical  decisions  of  the  National  Aeronautics  and  Space  Administration,  the 
Department  of  Energy,  and  the  Department  of  Defense  (Keefer  et  al,  2000). 

VFT  is  a  decision  analysis  technique  that  provides  an  objective  view  of  a 
subjectively-based  decision.  Even  though  this  concept  is  not  new,  it  will  be  applied 
differently  in  this  thesis.  Normally,  the  VFT  approach  is  used  to  guide  and  solve  a 
decision  problem  such  as  utility  privatization  evaluation.  However,  in  this  thesis,  the 
VFT  approach  will  be  used  to  identify  both  new  and  existing  values  associated  with  the 
utility  system  evaluation  and  privatization  process.  These  values  will  be  incorporated 
into  a  quantitative  structure  or  VFT  model,  which  will  be  used  to  evaluate  various 
alternatives  for  a  privatized  utility  system.  The  VFT  methodology  consists  of  the  ten-step 
process  outlined  by  Shoviak  (2001). 

1.6  Research  Scope  and  limitations 

This  research  will  focus  on  providing  insight  to  decision  makers  responsible  for 
selecting  and  evaluating  privatized  utility  providers  to  maintain  Air  Force  utility  systems. 
However,  there  are  two  primary  limitations  associated  with  this  research.  First,  this 
research  deviates  from  the  traditional  VFT  hierarchy  weighting  step.  Since,  the 
researcher  initially  generated  suggested  weights  for  each  of  the  values  and  the  decision 
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maker  adjusted  them  accordingly.  Second,  sufficient  real  world  data  to  validate  the 
model  is  unavailable;  therefore  this  thesis  will  use  notional  data  to  validate  the  audit 
model. 

1.7  Document  Structure 

The  literature  review,  presented  in  Chapter  2,  provides  an  in-depth  examination  of 
relevant  literature  and  defines  a  focus  (basis)  for  the  reader  regarding  the  questions  posed 
in  this  introduction.  Chapter  2  also  introduces  the  decision  analysis  methodology  and 
defines  the  VFT  process.  A  presentation  of  the  value  of  a  utility  system,  how  this  value 
is  measured  using  VFT,  and  how  it  can  be  applied  to  decision  makers  evaluating  utility 
privatization  is  provided  in  Chapter  3.  Also,  Chapter  3  provides  a  step-by-step 
breakdown  of  values  and  measures  used  in  the  evaluation  model.  Data  Analysis,  Chapter 
4,  presents  the  results  from  the  model’s  detenninistic  and  sensitivity  analysis.  Chapter  5 
presents  a  summary  of  the  research  results  and  offers  conclusions  and  recommendations 
regarding  the  utility  privatization  process 
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Chapter  2.  Background  of  the  Problem/Literature  Review 


This  chapter  explains  the  concept  of  utility  privatization  and  discusses  the  need  to 
evaluate  its  effectiveness.  The  chapter  initially  establishes  a  working  definition  for 
privatization  and  provides  a  brief  history  of  its  use.  It  then  defines  the  term  utility  system 
and  establishes  a  more  specific  working  definition  for  utility  privatization.  This 
definition  is  followed  by  a  brief  history  of  utility  privatization  efforts  in  the  Air  Force. 
With  this  foundation  established,  the  chapter  examines  the  current  laws  and  directives 
that  guide  Air  Force  utility  privatization  and  reviews  the  general  contracting  process.  To 
help  justify  this  research,  the  chapter  explores  the  current  privatization  debate  and 
reviews  previous  research  in  privatization  evaluation.  The  chapter  concludes  by 
presenting  a  ten-step  value-focused  thinking  process  for  conducting  this  type  of  decision 
analysis;  it  describes  what  is  involved  in  the  methodology  and  how  it  can  be  used  in  this 
research. 

2.1  Privatization  in  General 

Privatization  is  a  management  practice  used  throughout  the  world  to  lower 
operating  costs  in  organizations  and  governments  (Savas,  2001).  Privatization  can  take 
many  forms;  it  can  represent  the  complete  removal  of  government  from  the  production 
and  delivery  of  services  or  it  can  simply  mean  to  outsource  (contract  out)  (Greene,  1996). 
Many  governments  in  developed  and  developing  countries  are  using  this  management 
initiative  to  reinvent  and  reengineer  their  government  to  achieve  the  effectiveness  and 
efficiency  of  businesses  in  the  private  sector  (Korosec  et  al,  1996). 
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2.1.1  Privatization  Defined 


The  concept  of  privatization  and  its  use  can  be  difficult  to  define.  Therefore,  it  is 
very  important  to  understand  the  theory  behind  it.  Although  many  definitions  have  been 
used  to  describe  privatization,  Barnekov  et  al  (1990:136)  best  summarized  it  by 
explaining: 

. . .  Part  of  the  problem  of  any  discussion  of  privatization  is  that  the 
meaning  of  the  tenn  is  confusing  because  it  has  been  used  to  refer  to 
several  types  of  policy  initiatives,  this  is  to  include  the  shift  from  public  to 
private  provision  of  goods  or  services  (through  contracting  out  or  voucher 
arrangements.... 

Management  guru  Peter  Drucker  coined  the  privatization  term  in  the  late  1960s. 

He  argued  that  government  was  good  at  making  decisions,  but  bad  at  executing 
them  (Hodge,  2000: 13).  Therefore,  he  contended  that  the  execution  of 
government  services  should  be  separated  from  public  policy  and  “reprivatized” 

(Hodge,  2000: 13).  Since  then,  Savas  (1987:3)  expanded  Drucker’s  definition  by 
stating  “. .  .privatization  is  the  act  of  reducing  the  role  of  government,  or 
increasing  the  role  of  the  private  sector,  in  an  activity  or  in  the  ownership  of 
assets. . .”  Savas  (2001)  further  expanded  the  tenn’s  meaning  by  stating  that 
privatization  means  having  greater  reliance  on  private  institutions  in  the  civilian 
sector  and  less  dependence  on  government  to  satisfy  the  important  needs  of 
society.  For  the  purposes  of  this  research,  privatization  will  be  defined  as 
“conveying  or  transferring  responsibility  for  a  government  function  to  the  civilian 
sector  in  order  to  provide  better  efficiency.” 
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2.1.2  History  of  Privatization 


Privatization  is  not  a  new  concept  in  the  United  States;  it  is  as  old  as  the  federal 
government.  Governments  have  always  hired  private  businesses  to  manage  or  deliver 
services  through  a  contract.  For  example,  the  nineteenth  century  gave  us  the  pony 
express  as  an  example  of  privatization,  although  it  was  not  labeled  as  such  at  the  time 
(Johnson  et  al,  2000:2).  For  centuries,  federal  and  state  governments  have  used  private 
builders  to  construct  and  maintain  roads,  streets,  and  highways  (Johnson  et  al,  2000:2). 
Clearly,  history  has  shown  that  the  federal  government  has  relied  on  the  private  sector  to 
provide  important  necessities  from  time  to  time. 

For  more  than  half  a  century,  privatization  slowly  integrated  itself  into  federal 
government  processes.  In  1955,  President  Eisenhower  first  applied  it  in  the  Bureau  of  the 
Budget  by  establishing  a  policy  of  increasing  reliance  on  the  private  sector  for  goods  and 
services  (Wheeler,  1987:30).  In  1966,  privatization  was  further  incorporated  in  the 
federal  government  during  President  Johnson’s  administration  when  the  Bureau  of  the 
Budget  issued  Circular  A-76,  the  first  authoritative  guide  for  all  privatization  initiatives 
in  the  federal  government  (Pope,  1990:9).  This  document  was  designed  to  increase 
efficiency  in  producing  government-financed  commercial  services  through  the  promotion 
of  better  management  initiatives  and  fair  competition.  Also,  the  document  provides 
guidance  for  distinguishing  between  “inherently  government  functions”  and  other 
functions,  which  can  be  contracted  out  (Prager  et  al,  1996:187). 

In  the  late  1960s  and  early  1970s,  support  for  privatization  increased  due  primarily 
to  the  American  voter’s  preference  for  lower  taxes  and  smaller  government  (Savas, 
2000:38).  As  a  result,  lower  tax  revenues  applied  pressure  to  privatize  in  order  for  the 
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federal  government  to  balance  limited  resources  with  the  demands  of  American  society. 
However,  the  federal  government  did  not  give  privatization  major  consideration  until  the 
1980s. 

In  the  1980s,  the  Reagan  Administration  and  the  conservative  movement  to  reduce 
the  size  of  government  created  a  major  impetus  for  implementing  privatization  (Johnson 
et  al,  2000:5).  Many  supporters,  both  in  and  out  of  Congress,  believed  privatization  to  be 
the  remedy  for  fiscal  pressure  because  of  the  lower  costs  assumed  with  the  private  sector 
(Greene,  1996).  The  supporters  of  privatization  believed  that  using  the  experience  of 
businesses  would  cause  the  federal  government  to  become  more  efficient. 

The  fall  of  the  Soviet  Union  and  the  end  of  the  Cold  War  were  responsible  for 
giving  privatization  a  major  thrust  in  the  1990s.  Faced  with  the  burden  of  a  shrinking 
military  budget,  the  Department  of  Defense  initiated  a  major  move  to  privatize  functions 
or  components  that  the  federal  government  had  historically  conducted  or  performed 
(Hargett,  2003:21).  These  components  included,  but  were  not  limited  to,  management 
responsibility;  assets  and  their  operation  and  maintenance;  personnel;  and  capital 
investments  for  upgrades,  renewals,  and  improvements  (Hargett,  2003:21).  By 
privatizing  service  functions  and  housing  assets,  the  Department  of  Defense  believed  the 
superior  market  strategies  of  the  private  sector  would  help  reduce  big  government  and 
eliminate  waste,  thus  achieving  greater  efficiency  (Hargett,  2003:21). 

2.2  Utility  Privatization 

As  competition  for  the  ever-shrinking  defense  budget  increased  during  the  1 990s, 
the  number  of  Air  Force  personnel  continued  to  decline  because  of  force  reduction  efforts 
and  massive  budget  cuts.  At  the  same  time,  the  aging  infrastructure  at  many  Air  Force 
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installations  was  placing  increased  demands  on  increasingly  scarce  operations  and 
maintenance  dollars  (James,  1999).  After  years  of  inadequate  funding  and  a  shortage  of 
trained  utility  system  technicians,  it  was  revealed  that  military  utility  systems  were  not 
meeting  industry  standards  (Robbins,  2001).  The  Department  of  Defense  believed  that 
by  taking  advantage  of  the  private  sector’s  efficiencies,  entrepreneurship,  economies  of 
scale,  innovations,  and  financing,  they  would  provide  military  installations  with  safe, 
reliable  energy  supplies  and  utility  services  essential  to  supporting  the  mission 
(Krachman  et  al,  2003:23). 

2.2.1  Utility  System  and  Utility  Privatization  Defined 

A  utility  system  can  be  defined  as  any  system  used  for  the  generation  of  electric 
power,  treatment  or  supply  of  water,  collection  or  treatment  of  wastewater,  and  supply  of 
natural  gas.  This  definition  includes  the  distribution  system,  equipment,  fixtures, 
structures,  and  other  improvements  to  the  utility  system.  A  member  of  an  Anny  utility 
privatization  team  best  defined  the  utility  privatization  concept  by  explaining  that  it  is  the 
transfer  of  the  distribution  system  to  include  “. .  .the  buildings,  the  pipes,  and  the  wires, 
but  not  the  energy  commodity  itself. . .”  (James,  1999).  Under  the  utility  privatization 
concept,  the  methodology  is  to  “convey  government-owned  utility  systems  to  the  private 
sector”  (Sayeed,  2002  :B-2).  According  to  the  Department  of  Defense,  this  partnership 
between  the  Air  Force  and  private  sector  makes  economical  and  operational  sense.  To 
state  it  more  simply,  utility  privatization  is  considered  to  be  getting  the  military  out  of  the 
utility  operation  business  and  more  into  the  energy  management  business. 
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2.2.2  History  of  Air  Force  Utility  Privatization 


The  infrastructure  at  many  Air  Force  installations,  which  includes  utility 
components  such  as  electrical,  natural  gas,  raw  water,  potable  water,  and  wastewater 
systems,  has  been  inadequately  funded  (Robbins,  2001).  This  has  prevented  military 
installations  from  being  able  to  upgrade  and  maintain  their  utility  systems.  This  lack  of 
attention  has  led  to  dilapidated  utility  systems  at  many  installations.  The  cost  to  upgrade 
and  repair  these  systems  is  currently  estimated  at  over  $4  billion  (Sayeed,  2002:  B-3). 
Therefore,  the  Department  of  Defense  and  the  Air  Force  are  looking  to  the  efficiencies 
produced  by  privatization  as  a  means  to  fund  and  improve  their  utility  systems.  The 
assumption  is  that  the  use  of  private  utility  companies  will  bring  military  utility 
infrastructures  up  to  current  codes  and  standards. 

2.2.3  Air  Force  Utility  Privatization  Laws  and  Directives 

Several  policies  are  responsible  for  charting  the  path  towards  Air  Force  utility 
privatization;  however,  the  four  most  important  ones  were  three  Defense  Reform 
Initiative  Directives  (DRIDs)  and  one  Legislative  Authority  Title,  10  United  States  Code 
Section  26888  (10  U.S.C.  26888).  For  utility  privatization  to  work,  the  guidance  from 
these  policies  must  be  fully  implemented  in  all  planning  stages.  These  policies  direct 
military  service  departments  to  reengineer  business  practices,  explain  what  utility 
systems  can  be  exempted,  explain  how  to  report  progress,  and  establish  milestones. 

The  DRIDs  consists  of  four  principle  areas  that  have  guided  and  shaped  the 
privatization  initiative  for  the  Department  of  Defense.  These  areas  are  defined  as 
reengineering,  consolidating,  competing,  and  eliminating.  The  goal  of  the  reengineering 
principle  is  to  urge  the  Department  of  Defense  to  adopt  modem  business  practices  to 
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achieve  world-class  standards  of  performances.  The  consolidating  principle  attempts  to 
streamline  organizations  in  the  Department  of  Defense  to  remove  redundancy  and 
maximize  synergy.  For  the  competing  principle,  market  mechanisms  are  applied  to 
business  practices  to  help  improve  quality,  reduce  costs,  and  respond  to  customer  needs. 
Finally,  the  eliminating  principle  strives  to  reduce  excessive  support  structures  to  free 
resources  and  focus  on  core  competencies. 

Issued  in  December  1997,  DRID  #9  (Privatizing  Utility  Systems)  was  the  first 
privatization-based  DRID.  This  reform  declared  that  the  Department  of  Defense  would 
privatize  all  utility  systems,  except  those  needed  for  unique  security  reasons  or 
considered  uneconomical  to  privatize,  by  1  January  2000.  The  military  departments  were 
directed  to  present  their  strategy  for  privatization  to  the  Defense  Management  Utility 
Privatization  Council  no  later  than  13  March  1998  (Deputy  of  Secretary  of  Defense — 
DRID  #9,  1997). 

DRID  #21  (Formation  of  the  Defense  Energy  Support  Center)  was  issued  a  year 
later.  This  directive  was  responsible  for  the  formation  of  the  Defense  Energy  Support 
Center  (DESC)  (Deputy  of  Secretary  of  Defense — DRID  #21,  1998).  According  to  this 
directive,  the  DESC  is  responsible  for  assisting  the  Air  Force  with  contracting  actions  in 
support  of  utility  privatization  (Deputy  of  Secretary  of  Defense — DRID  #21,  1998). 

Finally,  DRID  #49  (Privatizing  of  Utility  Systems)  was  issued  in  December  1998. 
This  directive  established  quarterly  reporting,  implementation  plans,  and  milestones  for 
utility  privatization  (Deputy  of  Secretary  of  Defense — DRID  #49,  1998).  To  ensure 
progress  towards  the  privatization  goal,  three  milestones  were  established.  The  first 
milestone  was  30  September  2000;  it  required  the  completion  of  a  “go/no-go” 
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determination  as  to  which  utility  systems  would  privatization  be  pursued.  For  those 
systems  that  were  considered  appropriate  for  privatization  consideration,  the  second 
milestone  required  solicitations  to  be  released  no  later  than  30  September  2001.  The  last 
milestone  required  installations  to  make  plans  to  accommodate  the  award  of  privatization 
contracts  no  later  than  30  September  2003  (Deputy  of  Secretary  of  Defense — DRID  #49, 
1998). 

In  November  1998,  the  Legislative  Authority  Title  10  USC  Section  2688  was 
passed.  Part  (a)  of  this  legislation  detailed  authority  for  utility  conveyance  in  the 
Department  of  Defense  (Congress,  1997).  This  legislation  allowed  military  department 
secretaries  to  convey  utility  systems,  which  are  not  core  mission  requirements.  Also,  the 
legislation  explains  the  utility  system  conveyance  selection  process  as  well  as  payment 
treatments  to  the  privatized  utility  provider.  This  legislation  was  later  amended  in  fiscal 
year  2000  with  the  National  Defense  Authorization  Act.  This  act  extended  the  authority 
for  military  departments  to  enter  utility  system  service  contracts  for  up  to  50  years  and 
the  use  of  the  Military  Construction  (MILCON)  program  to  fund  utility  privatization 
initiatives. 

2.2.4  Contracting  Process 

As  mentioned  in  DRID  #49,  the  Secretary  of  Defense  established  milestones  for 
accomplishing  privatization  actions  (Deputy  of  Secretary  of  Defense — DRID  #49,  1998). 
As  a  result,  the  Air  Force  Utilities  Privatization  Process  was  created  to  help  analyze  and 
track  all  Air  Force  utility  systems  being  considered  for  privatization.  According  to  the 
Air  Force  Civil  Engineer  Support  Agency  (AFCESA),  the  entire  process  takes 
approximately  two  years  from  start  to  finish  (AFCESA  Home  Page,  2003).  This  process 
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has  three  phases  that  facilitate  the  transition  of  Air  Force  utility  systems  to  the  private 
sector:  Phase  I,  Projection  Plan  and  Feasibility  Analysis;  Phase  II,  Comprehensive 
Analysis;  and  Phase  III,  Implementation. 

2. 2. 4.1  Phase  I  -  Feasibility  Analysis  and  Market  Review 

The  first  phase  of  the  utility  privatization  process  is  composed  of  two  main 
components — feasibility  analysis  and  market  review.  These  components  are  responsible 
for  helping  establish  the  “go/no-go”  decision  before  an  installation’s  utility  system 
proceeds  with  Phase  II  actions  or  seeks  exemption  from  privatization.  During  the 
feasibility  component  of  Phase  I,  a  preliminary  analysis  is  used  to  determine  if  a  utility 
system  should  be  exempted.  DRID  #49  exempts  utility  systems  that  are  uneconomical  to 
privatize  or  have  unique  security  reasons  (Deputy  of  Secretary  of  Defense — DRID  #49, 
1998).  “Unique  security  reasons”  are  those  situations  in  which  ownership  of  the  system 
by  a  private  utility  or  other  entity  would  substantially  impair  the  mission  of  the 
department  concerned  or  would  compromise  operations  or  property  (Sayeed,  B-9:2002). 
A  utility  system  can  also  be  exempted  if  there  is  a  lack  of  interest  or  response  from  any 
utility  company  during  the  market  review  component.  The  market  review  component 
consists  of  a  military  installation  placing  an  announcement  of  the  intention  to  privatize 
their  utility  system  in  the  Federal  Business  Opportunities/Commerce  Business  Daily 
(FBO/CBD)  federal  government  solicitation  publication  (AFCESA  Home  Page,  2003). 

2. 2. 4. 2  Phase  II-  Comprehensive  Analysis 

The  second  phase  of  the  process  requires  the  installation  to  accomplish  an 
environmental  impact  assessment,  develop  real  estate  documents  for  easements,  and 
collect  technical  and  cost  data  (Sayeed,  D-2:2002).  A  life  cycle  cost  analysis  and  cost  of 
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service  study  is  also  completed  to  provide  the  information  necessary  to  help  the 
installation  with  its  source  selection  plan  (AFCESA  Home  Page,  2003).  If  more  than  one 
interested  provider  is  identified  in  Phase  I,  the  Federal  Acquisition  Regulation  requires 
the  Air  Force  to  conduct  a  full  and  open  competition.  Consequently,  nonnal  competitive 
utility  service  contract  procedures  must  be  implemented  when  establishing  the  evaluation 
criteria  and  drafting  the  Request  for  Proposal  (RFP). 

2. 2.4. 3  Phase  III  -  Implementation 

During  the  third  phase,  the  RFP  is  finalized  and  issued  by  the  installation’s 
Contracting  Squadron  (Sayeed,  D-2:2002)  .  The  final  RFP  is  placed  in  the  FBO/CBD 
federal  government  solicitation  publication,  and  all  ensuing  proposals  are  reviewed  and 
evaluated  by  a  source  selection  board  using  technical  capability,  past  performance,  and 
cost/price  as  determined  by  the  evaluation  criteria.  Negotiations  are  then  conducted  with 
potential  utility  providers  before  a  recommendation  is  made  to  the  installation.  The 
recommendation  by  the  source  selection  board  will  be  for  a  provider  who  can  give  the 
best  technical  support  to  the  installation  at  a  cost  equal  to  or  below  the  independent 
government  estimate  (AFCESA  Home  Page,  2003)  .  Once  the  appropriate  higher 
headquarters  reviews  the  privatization  packages,  they  are  forwarded  to  the  Secretary  of 
the  Air  Force  for  approval  of  award  (AFCESA  Home  Page,  2003).  After  the  contract  had 
been  awarded,  the  installation  begins  the  process  of  transferring  responsibility  of  the 
utility;  the  utility  provider  then  begins  to  operate,  maintain,  upgrade,  and  improve  the 
system  to  industry  standards  for  the  installation.  However,  the  Air  Force’s  responsibility 
does  not  end  with  the  award  of  the  contract.  Since  utilities  are  critical  to  the  operation 
and  readiness  of  the  Air  Force’s  mission,  the  installation  and  the  Air  Force  still  have  a 
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responsibility  to  monitor  the  perfonnance  of  contracted  utility  providers  (AFCESA  Home 
Page,  2003). 

2.3  Privatization  Debate 

Even  though  privatization  is  very  common  in  governments  around  the  globe,  there 
is  still  considerable  opposition  to  its  use.  According  to  Savas  (1982:89),  “...much 
debate,  with  a  great  deal  of  heat  but  relatively  little  light. .  .has  been  generated  on  the 
issue  of  which  is  best. .  .public  or  private  production  of  service.”  Since  the  boom  of  the 
privatization  movement  in  the  early  1990s,  there  have  been  significant  community 
concerns  about  its  effectiveness  (Hodge,  2000:8).  Concerns  in  the  areas  of  performance 
and  economics  are  just  some  of  the  issues  contributing  to  the  debate  (Hodge,  2000:8). 

Opponents  of  privatization  argue  that  it  is  naive  to  believe  that  privatization  will 
decrease  operating  costs.  They  support  this  statement  by  explaining  that  organizations 
that  only  focus  on  the  expected  “cost  savings  and  efficiency  improvements  of 
privatization  overlook  the  tendency  of  private  providers  to  service  only  the  easy  and 
profitable  customers,  while  the  difficult  and  unprofitable  are  neglected”  (Barnekov  et  al, 
1990:137).  Furthermore,  they  believe  opportunities  for  bribery  or  kickbacks  are  created 
when  the  government  allows  services  and  functions  to  be  performed  by  the  private  sector 
(Bamekov  et  al,  1990:138).  To  prevent  these  outcomes,  opponents  believe  that 
governments  should  allocate  resources  to  “regulate  and  monitor”  contractors  providing 
services  and  goods  for  the  federal  government  (Barnekov  et  al,  1990:138). 

On  the  other  hand,  privatization  supporters  dismiss  the  critic’s  concerns  by  arguing 
that  privatization  provides  better  services  and  improves  accountability;  they  point  out  that 
“careful  writing  of  contracts  and  monitoring”  can  ensure  that  the  private  sector  efficiently 
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provides  services  to  governments  with  minimal  discrepancies  (Barnekov  et  al,  1990:138). 
The  following  list  summarized  by  Savas  (1982:89)  best  presents  the  arguments  in  favor 
of  privatization: 

1)  Contracting  is  more  efficient  because 

(a)  It  harnesses  competitive  forces  and  brings  the  pressure  of  the 
marketplace  to  bear  on  inefficient  producers. 

(b)  It  permits  better  management,  free  of  most  of  the  distractions 
characteristics  of  overtly  political  organizations. 

(c)  The  costs  and  benefits  of  managerial  decisions  are  felt  more 
directly  by  the  decision  maker,  whose  own  rewards  are  directly  at 
stake. 

2)  Contracting  makes  it  possible  for  government  to  take  advantage  of 
specialized  skills  lacking  in  its  own  work  force;  it  overcomes  obsolete 
salary  limitations  and  antiquated  civil  service  restrictions. 

3)  Contracting  allows  flexibility  in  adjusting  the  size  of  a  program  up  or 
down  in  response  to  changing  demand  and  to  changing  availability  of 
funds. 

4)  Contracting  pennits  a  quicker  response  to  new  needs  and  facilitates 
experimentation  in  new  programs. 

5)  Contracting  is  a  way  of  avoiding  large  capital  outlays;  it  spreads  costs 
over  time  at  a  relatively  constant  and  predictable  level. 

6)  Contracting  pennits  economies  of  scale  regardless  of  the  scale  of  the 
government  entity  involved. 

7)  Contracting  a  portion  of  the  work  offers  a  yardstick  for  comparison; 
the  cost  of  the  service  is  highly  visible  in  the  price  of  the  contract, 
unlike  most  government  services. 

8)  Contracting  can  reduce  dependence  on  a  single  supplier  (a  government 
monopoly)  and  so  lessens  the  vulnerability  of  the  service  to  strikes, 
slowdowns,  and  inept  leadership. 

9)  Contracting  limits  the  size  of  govermnent,  at  least  in  terms  of  the 
number  of  employees. 

Nevertheless,  both  sides  agree  that  constant  monitoring  and  evaluation  of  privatization 
efforts  can  be  quite  beneficial 

The  opposition’s  clear  discontent  with  privatization  is  evident  throughout  the 
world.  In  New  Zealand,  there  have  been  complaints  over  the  Department  of  Labour’s 
implementation  of  privatization  actions  (Hodge,  2000:8).  Similarly,  interest  groups  in  the 
United  Kingdom  have  expressed  their  dissatisfaction  with  the  privatization  program  of 
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the  Thatcher  Conservative  administration.  Former  British  Prime  Minister  Harold 
Macmillan  best  characterized  the  growing  British  opposition  to  privatization  by 
describing  it  as  “selling  the  family  silver”  (Hodge,  2000:8).  Although  the  previous 
examples  summarize  the  opposition’s  attitude  toward  privatization,  the  growing  debate  in 
the  state  of  Massachusetts  provides  the  basis  for  understanding  the  opposition’s 
argument. 

In  1997,  Wallin  analyzed  the  privatization  dispute  in  Massachusetts  between 
Governor  Weld’s  initiatives  and  the  legislative  body  representing  the  interests  of  state 
employees.  Wallin’s  analysis  of  the  state’s  privatization  program  found  that  the  Weld 
administration’s  eagerness  to  privatize  state  services  produced  many  errors.  In  particular, 
Wallin  highlighted  the  state’s  failure  to  adequately  measure  and  document  the  cost  and 
performance  variables  of  proposed  privatization  initiatives  before  privatizing  them.  As  a 
consequence,  the  state  of  Massachusetts  did  not  receive  the  projected  cost  savings  and 
performance  levels  as  predicted.  Wallin  explains,  “. .  .cost  and  performance  must  be 
carefully  measured  before  privatization  so  that  proper  evaluation  of  privatization’s  effects 
can  be  made”  (Wallin,  1997:1 1).  Wallin’s  analysis  emphasizes  the  importance  of 
viewing  privatization,  like  any  new  government  policy,  as  an  experiment  (Wallin, 

1997:1 1).  For  that  reason,  Wallin  urged  organizations  to  use  independent  checks  on 
privatization  decisions  and  constant  monitoring  of  privatized  service  providers  in  order  to 
improve  the  privatization  process  as  well  as  to  prevent  problems  like  those  experienced 
by  the  state  of  Massachusetts. 

The  Massachusetts’  privatization  debate  forced  Governor  Weld’s  administration  to 
revise  its  privatization  guidelines.  In  November  1993,  Weld’s  Office  of  Administration 
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and  Finance  released  a  seven-point  guide  for  “mitigating  imperfect  conditions”  of 
privatization  (Wallin,  1997:12).  The  following  strategy  from  the  guideline  emphasizes 
the  need  for  oversight  and  continuous  monitoring  of  the  privatization  process:  .  .to 
ensure  quality  and  responsiveness  -  develop  reliable  measures  of  service  quality, 
strengthen  in-house  monitoring  capacity,  and  write  contracts  with  periodic  performance 
reporting...”  (Wallin,  1997:12). 

Wallin  suggests  that  the  lessons  learned  from  Massachusetts’  privatization 
experience  could  provide  useful  information  for  organizations  new  to  the  world  of 
privatization.  After  much  criticism  and  learning  from  their  mistakes,  the  state  of 
Massachusetts  was  forced  to  adopt  a  bill  regulating  the  state’s  privatization  process 
(Wallin,  1997:1).  According  to  Wallin,  the  “inherent  difficulty  in  measuring 
performance  in  most  government  services  and  in  documenting  cost  savings  from  a 
change  of  service  provider  is  a  strong  argument  for  careful  consideration  of  privatization 
initiatives”  (Wallin,  1997:17).  Therefore,  Wallin  stresses  that  careful  independent  checks 
on  the  cost  and  performance  variables  of  privatization  decisions  can  prevent  attempts  to 
“stack  the  deck”  in  favor  of  privatization,  whether  intentional  or  not;  avoid  backlash  from 
the  critics;  and  provide  legitimacy  to  the  process  (Wallin,  1997:16).  The  problems 
experienced  by  Massachusetts  and  other  government  entities  indicate  that  the  Air  Force 
should  evaluate  its  utility  privatization  process  to  prevent  similar  problems. 

2.4  Previous  Research 

The  literature  review  indicates  a  research  gap  in  the  measurement  and  evaluation  of 
privatization  effectiveness.  Hodge  (2000:7)  attempted  to  fill  this  gap  by  synthesizing 
empirical  data  on  existing  privatization  studies  from  the  previous  20  years.  One  of 
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Hodge’s  research  questions  was  very  similar  to  a  question  of  this  research.  His  question 
states,  “Does  privatization  usually  improve  service  provision,  and  at  lower  cost,  or  are 
such  generalities  misleading  and  inappropriate?”  Since  the  scope  of  Hodge’s  research  is 
somewhat  similar  to  the  scope  of  this  research  effort,  and  to  avoid  research  redundancy, 
the  literature  review  will  delve  into  Hodge’s  analysis  in  great  detail. 

Using  internationally  recognized  privatization  goals  and  other  suggested  goals  from 
the  public  sector,  Hodge  (1999:457)  identified  five  different  areas  or  dimensions  to  be 
used  to  construct  the  framework  for  making  privatization  decisions  on  government 
services.  These  five  dimensions  were  economic  performance,  social  performance, 
democratic  perfonnance,  legal  performance,  and  political  performance.  The  economic 
and  social  performance  dimensions  were  highly  relevant  to  this  research  effort.  The 
economic  dimension  uses  several  economic  indicators  to  look  at  the  areas  of  economic 
efficiency,  financial  returns,  and  economy  (Hodge,  1999:458).  The  social  dimension  is 
responsible  for  reviewing  “the  promised  benefits  to  the  community”  of  lower  prices  and 
equal  or  better  service  (Hodge,  1999:458). 

Hodge  used  the  meta-analysis  review  technique  to  analyze  129  studies  possibly 
containing  privatization  empirical  evidence.  According  to  Hodge  (1999:459),  “this 
technique  uses  as  its  data  the  statistical  measurements  found  in  all  available  reports  that 
have  investigated  the  effectiveness  of  contracting.”  Normally,  this  type  of  research  will 
use  the  t-test  to  detennine  whether  the  mean  of  a  test  group  is  statistically  different  from 
the  control  group.  The  t-test  can  help  distinguish  a  possible  relationship  between  the  two 
groups  in  terms  of  “an  estimate  of  the  magnitude  of  the  relationship  (the  effect  size)  and 
an  indication  of  the  accuracy  or  reliability  of  the  estimated  effect  size  (as  in  a  confidence 
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interval  placed  around  the  estimate)”  (Hodge,  1999:459).  Therefore,  the  meta-analysis 
technique  can  use  the  effect  size  variable  to  help  exclude  studies  with  deficiencies  that 
are  likely  to  distort  the  analysis’  outcome.  After  excluding  the  majority  of  studies 
because  of  insufficient  reporting  of  statistical  data,  only  28  remained  for  analysis. 

Hodge’s  (1999:460)  analysis  produced  over  66  effect-size  estimates  for  a  data  set 
spanning  1976  to  1994.  The  meta-analysis  concluded  that  the  largest  economic 
improvements  were  found  in  services  such  as  maintenance,  cleaning,  and  refuse 
collection.  These  services  showed  the  highest  effect  sizes,  ranging  from  19  to  30.5 
percent  (Hodge,  2000: 128).  As  for  the  social  performance,  the  analysis  indicated  that 
there  was  no  discemable  relationship  to  determine  if  privatization  reduces  or  increases 
the  service  level  (Hodge,  2000:156). 

Hodge’s  analysis  of  privatization  initiatives  assists  this  research  effort  in  many 
ways.  First,  it  details  the  contextual  background  of  the  community’s  expectations  (or 
values)  when  making  privatization  decisions,  thus  providing  this  research  with  values  that 
help  construct  the  framework  for  a  standardized  analysis  model  used  to  evaluate  utility 
privatization  effectiveness.  Second,  Hodge’s  analysis  gives  a  breakdown  of  international 
empirical  evidence  relevant  to  the  effectiveness  of  privatization.  Lastly,  the  analysis 
provides  a  wide  range  of  quantitative  data  in  relationship  to  privatization’s  effectiveness. 

However,  Hodge’s  analysis  also  fails  to  support  this  research  effort  in  many  other 
areas.  First,  Hodge’s  research  provides  very  little  data  on  the  effectiveness  of  utility 
privatization.  In  fact,  a  water  treatment  study  was  the  only  indication  of  a  utility 
privatization  study  being  used  in  his  analysis.  Second,  Hodge’s  research  fails  to  detail 
the  methodologies  used  by  the  public  and  private  sector  to  evaluate  the  effectiveness  of 
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privatization.  Lastly,  Hodge’s  analysis  fails  to  recommend  the  most  effective  method  to 
evaluate  the  effectiveness  of  privatization  initiatives.  Therefore,  this  research  will 
attempt  to  fill  the  gaps  in  Hodge’s  research  by  answering  the  investigative  questions  as 
stated  in  the  previous  chapter. 

2.5  Decision  Analysis 

Determining  whether  to  privatize  a  utility  system  at  an  Air  Force  installation,  and 
then  evaluating  the  effectiveness  of  that  decision,  is  hard  for  many  reasons.  For  starters, 
the  complex  nature  of  the  decision  makes  it  a  difficult  one.  Identifying  the  values 
associated  with  the  goals  of  privatization  and  organizing  them  are  the  primary  reasons  for 
the  complexity.  Decision  analysis  can  provide  an  effective  method  for  the  Air  Force  to 
structure  this  complex  problem  for  analysis  (Clemen  et  al,  2001:2).  Similarly,  the 
competing  and  subjectively  based  multiple  objectives  are  another  reasons  for  this 
decision’s  difficulty.  Again,  decision  analysis  is  a  tool  the  Air  Force  can  use  to  clearly 
establish  the  variables  and  objectives  of  utility  privatization,  thereby  making  it  less 
confusing  to  stakeholders  involved  in  the  decision. 

2.5.1  Introduction  to  Decision  Analysis 

Decision  analysis  uses  a  set  of  quantitative  methods  to  analyze  and  make  decisions. 
These  quantitative  methods  are  designed  to  help  the  decision  maker  systematically  make 
better  decisions.  As  a  result,  the  decision  making  process  becomes  more  nonnative, 
rather  than  descriptive.  Furthermore,  the  structuring  tools  of  decision  analysis  (such  as 
influence  diagrams,  value  hierarchies,  and  decision  trees)  provide  the  decision  maker 
with  indispensable  evaluation  insight  (Clemen  et  al,  2001 :2).  In  summary,  decision 
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analysis  allows  the  decision  maker  to  translate  hard-to-define  goals  and  measures  of  a 
decision  into  a  clear,  defendable  structure  for  better  insight  and  facilitation. 

2.5.2  Alternative-Focused  Thinking  versus  Value-Focused  Thinking 

Normally,  there  are  two  approaches  that  can  be  applied  to  the  decision  analysis 
methodology:  Alternative-Focused  Thinking  (AFT)  and  Value-Focused  Thinking  (VFT). 
The  default  approach  for  many  decision  makers  is  the  AFT  approach.  As  illustrated  in 
Figure  1,  the  AFT  approach  first  identifies  the  potential  alternatives  of  a  decision  and 
then  evaluates  these  alternatives  based  on  the  objectives  and  criteria  of  the  decision. 


Figure  1.  Alternative  and  Value-Focused  Thinking  Approaches  (Clark,  2001:  2-36) 
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Keeney  (1994:33)  describes  the  AFT  approach  as  putting  the  cart  (alternatives) 
before  the  horse  (values).  He  argues  that  the  approach  may  or  may  not  solve  the  problem 
because  the  decision  maker  is  merely  selecting  the  “best”  of  readily  available  alternatives 
(Keeney,  1992:47).  In  other  words,  if  all  readily  available  alternatives  are  bad,  then  the 
AFT  approach  will  only  assist  the  decision  maker  in  selecting  the  “best”  of  the  worst 
alternatives,  thus  failing  to  solve  the  problem.  By  contrast,  the  VFT  approach  helps  solve 
the  decision  problem  by  establishing  a  list  of  alternatives  based  on  the  values  of  the 
decision  maker.  Therefore,  the  VFT  approach  uses  values  to  help  identify  alternatives 
appropriate  for  the  decision. 

2.6  Value-Focused  Thinking 

VFT  is  a  multiple  objective  decision  analysis  process  that  reverses  the  AFT 
approach  by  first  defining  the  values  that  are  important  to  the  decision  maker.  Numerous 
decision  makers  in  the  public  and  private  sectors  have  successfully  used  the  VFT 
approach.  In  2000,  Keefer  et  al  (2000: 12)  listed  the  applications  of  decision  analysis 
methods  from  1990  to  1999.  Their  literature  review  indicated  that  VFT  and  various  other 
decision  analysis  methods  have  been  successfully  used  in  a  wide  variety  of  areas: 
energy,  services  and  manufacturing,  medical,  military,  public  policy,  and  various  general 
categories.  The  authors  concluded  that  their  literature  review  indicated  that  decision 
analysis,  including  VFT,  is  a  commonly  used  approach  to  help  make  strategic  and  tactical 
decisions  throughout  the  world  (Keefer  et  al,  2000:28). 

Since  the  values  are  identified  and  structured  before  the  alternatives,  VFT  can  offer 
the  decision  maker  several  advantages.  First,  the  VFT  process  prompts  the  decision 
maker  to  clarify  the  problem.  When  problems  are  clearly  defined,  the  likelihood  of  the 
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decision  maker  solving  the  problem  increases.  Second,  VFT  can  improve  the  likelihood 
of  solving  the  decision  problem  by  identifying  value  conflicts.  The  explicit  nature  of 
values  structured  in  the  VFT  hierarchy  can  generate  discussions  that  “separate 
disagreements  about  possible  consequences  (values)  from  disagreements  about  the 
relative  desirability  of  those  consequences  (values)”  (Keeney,  1992:26).  Once  the  value 
conflicts  are  identified,  the  decision  maker  and  stakeholders  can  constructively  discuss 
how  to  reduce  them  (Keeney,  1992:26).  Finally,  VFT  compels  the  decision  maker  to  use 
the  values  in  a  consistent  manner.  In  VFT,  the  decision  maker  must  apply  all  relevant 
values  in  a  consistent  manner  to  properly  evaluate  alternatives.  However,  the  values  do 
not  have  to  be  weighted  equally,  just  applied  to  every  alternative  in  the  same  manner. 

This  in  turn  reduces  the  likelihood  of  creating  bias  in  the  evaluation  process  while 
creating  a  decision  evaluation  model  that  is  defendable  and  repeatable.  Figure  2  provides 
an  overview  of  these  benefits  and  others  as  defined  by  Keeney. 
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Figure  2.  Benefits  of  Value-Focused  Thinking  (Clark,  2001:  2-36) 

2.7  Ten-Step  Value-Focused  Thinking  Process 

The  VFT  process  is  mainly  derived  from  the  methodology  instituted  by  Keeney 
(1992)  and  Kirkwood  (1997).  Shoviak  (2001 :63)  incorporated  the  ideas  and  principles  of 
the  previous  authors  into  the  ten-step  process  shown  in  Figure  3.  The  remainder  of  this 
section  will  discuss  these  steps  and  examine  their  applicability  to  this  research  effort. 
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Figure  3.  VFT:  Ten-Step  Process  (Shoviak  et  al,  2001) 


2.7.1  Step  1  -  Problem  Identification 

Before  VFT  can  begin,  the  decision  maker  must  clearly  define  the  exact  nature  of 
decision  problem.  According  to  Keeney  (1992:55),  the  decision  maker  usually  fails  to 
fully  understand  the  problem  and  the  objectives  used  to  solve  the  problem.  This  failure 
not  only  results  in  wasted  effort  and  resources;  it  also  renders  the  model  useless. 
Therefore,  it  is  very  important  that  the  decision  maker,  and  any  other  stakeholders 
associated  with  the  problem,  give  the  appropriate  time  and  effort  to  fully  identify  and 
define  the  decision  problem.  To  do  this,  the  decision  maker  should  ask  the  following 
questions  to  begin  constructing  a  value  hierarchy:  What  is  important  to  me  in  terms  of 
this  decision?  What  is  it  that  I  value  in  a  solution? 
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The  answers  to  these  questions  help  ensure  proper  identification  of  the  decision 
problem  by  forcing  the  decision  maker  to  examine  the  decision  context,  objective,  and 
direction/preference  (Weir,  2003).  The  decision  context  is  the  setting  in  which  the 
decision  occurs.  For  instance,  a  decision  maker  having  an  old  and  unreliable  truck  with 
constant  costly  repairs  would  be  an  example  of  a  decision  context  or  a  decision  setting. 
Accordingly,  the  decision  maker’s  objective  would  be  to  replace  the  truck.  In  addition, 
replacing  the  old  truck  with  a  new  truck  would  be  an  example  of  the  decision  maker’s 
direction/preference.  Thus,  the  fundamental  objective  for  this  particular  decision 
problem  would  be  to  “Buy  the  Best  Truck.” 

2.7.2  Step  2  -  Value  Hierarchy  Construction 

Once  the  fundamental  objective  is  defined,  the  construction  of  the  value  hierarchy 
can  begin.  Structuring  the  values  of  the  decision  maker  into  a  hierarchical  format  creates 
the  value  hierarchy.  This  format  serves  as  a  graphical  representation  of  the  values 
important  to  the  decision  problem  facing  the  decision  maker.  Thus,  the  graphical  format 
allows  the  decision  maker  and  stakeholders  to  visualize  how  their  values  impact  the 
decision  making  process.  Thus,  the  decision  maker  is  able  to  identify  missing  values  or 
“holes”  in  the  value  hierarchy  (Keeney,  1992:69).  Ultimately,  the  goal  is  to  fill  in  as 
many  “holes”  as  possible  in  order  to  construct  a  clear  and  comprehensive  value  hierarchy, 
which  will  aid  in  the  defensibility  and  facilitation  of  the  VFT  process. 

2. 7.2.1  Desired  Properties  of  Value  Hierarchies 

According  to  Kirkwood  (1997:16),  value  hierarchies  should  include  the  following 
desirable  properties:  completeness,  nonredundancy,  decomposability,  operability,  and 
small  size.  When  each  tier  of  a  value  hierarchy  is  “collectively  exhaustive,”  the  value 
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hierarchy  is  considered  to  encompass  completeness.  To  achieve  this  property,  the 
decision  maker  must  ensure  the  value  hierarchy  adequately  covers  all  values  important  to 
the  decision  problem  (Kirkwood,  1997:16).  Furthermore,  the  evaluation  measures  for  the 
lowest-tier  must  “adequately  measure  the  degree  of  attainment  of  their  associated 
objectives”  (Kirkwood,  1997:16).  By  ensuring  completeness  in  the  value  hierarchy, 
decision  alternatives  are  adequately  evaluated  and  ranked  accordingly. 

The  principle  of  nonredundancy  is  the  second  desired  property  of  value  hierarchies. 
When  value  hierarchies  are  nonredundant,  the  values  in  the  hierarchies  are  considered 
“mutually  exclusive”  (Kirkwood,  1997:17).  This  means  that  none  of  the  values  in  any 
given  tier  overlap.  When  evaluation  considerations  overlap,  objectives  are  subjected  to 
being  “double  counted,”  thus  giving  certain  objectives  “more  weight  than  was  intended 
when  the  weights  were  assigned  to  the  various  evaluation  measures”  (Kirkwood, 
1997:17).  The  application  of  nonredundancy  in  the  value  hierarchy  prevents  the  overlap 
of  evaluation  considerations. 

The  third  desired  property  of  value  hierarchies  is  decomposability  or 
“independence.”  This  principle  ensures  that  the  score  of  one  value’s  measure  is  not 
dependent  upon  the  score  of  another  value’s  measure.  For  instance,  Kirkwood’s 
example  uses  a  decision  maker  with  a  value  hierarchy  consisting  of  the  following  values: 
salary,  pension  benefits,  and  medical  coverage  (Kirkwood,  1997:17).  Despite  the 
appearance  of  nonredundancy,  the  value  hierarchy  may  still  lack  decomposability.  For 
example,  the  value  of  an  additional  $5,000  increase  in  salary  may  depend  on  pension 
benefits.  The  $5,000  increase  may  not  be  as  valuable  to  a  person  with  good  pension 
benefits  as  it  is  to  a  person  with  poor  pension  benefits  who  can  use  the  additional  $5,000 
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to  build  his/her  own  retirement  plan.  Consequently,  the  lack  of  decomposability  of 
values  makes  the  value  hierarchy  more  complex  (Kirkwood,  1997:18). 

The  fourth  desired  property  of  value  hierarchies  can  be  defined  as  operability.  The 
application  of  this  principle  ensures  that  the  value  hierarchy  is  understandable  to  the 
stakeholders  who  will  use  the  model  (Kirkwood,  1997:18).  The  operability  principle 
assists  the  value  hierarchy  in  better  facilitating  communication  and  improving  its 
defensibility. 

Small  size  is  the  fifth  and  last  desired  property  of  value  hierarchies.  This  principle 
stresses  the  need  to  make  the  value  hierarchy  as  small  as  possible  while  balancing 
defensibility  and  practicality.  Smaller  value  hierarchies  are  easier  to  communicate  to 
stakeholders  and  use  fewer  resources.  The  “test  of  importance”  ensures  a  small  hierarchy 
size  by  filtering  out  superfluous  values;  it  states  that  an  “evaluation  consideration  should 
be  included  in  a  value  hierarchy  only  if  possible  variations  among  the  alternatives  with 
respect  to  the  proposed  evaluation  consideration  could  change  the  preferred  alternative” 
(Kirkwood,  1997:19).  In  other  words,  the  test  identifies  values  that  do  not  contribute  to  a 
difference  in  the  top  ranked  alternative  (Kirkwood,  1997:18-19). 

2. 7.2.2  Generation  of  Values 

Normally,  the  decision  maker  is  ultimately  responsible  for  specifying  the  objectives 
or  values  important  to  the  decision  problem.  However,  the  decision  maker  should  also 
solicit  input  from  “individuals  interested  in  and  knowledgeable”  (i.e.,  stakeholders)  about 
the  decision  problem  (Keeney,  1992:56).  As  recommended  by  Kenney  (1992:56),  “the 
most  obvious  way  to  identify  objectives  is  to  engage  in  a  discussion  of  the  decision 
situation.”  Keeney  (1994:35)  identified  the  following  list  of  techniques  to  help  identify 
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objectives  and  recommended  the  use  of  a  facilitator.  The  questions  after  each  technique 
are  used  to  guide  the  decision  maker  during  the  process  (Shoviak,  2001 :48). 

1 .  Develop  a  wish  list.  What  do  you  want?  What  do  you  value?  What 
should  you  want? 

2.  Identify  alternatives.  What  is  a  perfect  alternative,  a  terrible 
alternative,  and  a  somewhat  reasonable  alternative? 

3.  Consider  problems  and  shortcomings.  What  needs  fixings? 

4.  Predict  consequences.  What  has  occurred  that  was  good  or  bad? 

What  might  occur  that  you  care  about? 

5.  Identify  goals,  constraints,  and  guidelines.  What  are  your 
aspirations?  What  limitations  are  place  on  you? 

6.  Consider  different  perspectives.  What  would  your  competitor  or 
constituency  be  concerned  about?  At  sometime  in  the  future,  what 
would  concern  you? 

7.  Determine  strategic  objectives.  What  are  your  ultimate  objectives? 

What  are  your  values  that  are  absolutely  fundamental? 

8.  Determine  generic  objectives.  What  objectives  do  you  have  for  your 
customers,  your  employees,  your  shareholders,  and  yourself?  What 
environmental,  social,  economic,  or  health  and  safety  objectives  are 
important? 

In  addition  to  Keeney’s  list  of  techniques,  the  gold,  silver,  and  platinum  standards 
are  supplementary  methods  used  to  help  the  decision  maker  and  stakeholders  generate 
values/objectives.  The  gold  standard  deductively  develops  the  value  model  by  examining 
an  organization’s  strategic  objectives,  vision,  or  plan.  The  organization’s  senior 
leadership  then  validates  the  constructed  hierarchy.  The  silver  standard  provides  a 
simpler  and  more  logical  value  model  than  the  gold  standard.  With  the  silver  standard, 
discussions  with  a  large  number  of  stakeholders  are  used  to  generate  values/objectives. 
During  these  group  discussions,  affinity  diagrams  are  used  to  help  inductively  build  the 
value  model.  Interviews  with  senior  leadership  and  key  technical  personnel  are  used  to 
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create  the  value  model  with  the  platinum  standard.  This  method  provides  a  more 
insightful  structure  because  of  the  stakeholders’  direct  involvement. 

2. 7.2.3  Structuring  the  Value 

A  value  hierarchy  is  constructed  by  structuring  the  values  of  a  decision  problem  in 
a  hierarchical  fashion.  At  the  very  top  of  the  value  hierarchy  is  the  overarching 
fundamental  objective.  The  fundamental  objective  is  then  divided  into  sub-objectives 
that  better  define  the  decision  problem.  When  sub-objectives  are  established  below  the 
fundamental  objective,  a  layer  or  tier  is  created.  As  the  value  hierarchy  structure  grows, 
the  values  of  the  lower-tiers  are  used  to  define  “the  important  attributes  of  those  values 
higher  in  the  hierarchy”  (Jurk,  2002:35).  Creation  of  tiers  in  this  manner  continues  until 
the  values  are  subdivided  to  a  level  at  which  measurement  and  evaluation  is  possible.  To 
assist  with  understanding  value  hierarchies,  consider  the  following  example  in  Figure  4. 
The  root  of  the  hierarchy  is  the  fundamental  objective,  “Buy  the  Best  Truck,”  with  first- 
tier  values  of  performance,  practicality,  and  safety  (Jurk,  2002:37). 
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Figure  4.  “Buy  the  Best  Truck”  Hierarchy  (Jurk,  200 1 :36) 


2.7.3  Step  3  -  Evaluation  Measure  Development 

When  the  values  can  no  longer  be  subdivided,  the  decision  maker  must  detennine 
the  type  of  measures  that  can  adequately  capture  the  values.  According  to  Kirkwood 
(1997:24),  an  evaluation  measure  specifies  the  .degree  of  attainment  of  [an] 
objective.”  Thus,  an  evaluation  measure  can  help  quantity  the  degree  of  attainment  by 
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allowing  an  “unambiguous  rating  of  how  well  an  alternative  does  with  respect  to  each 
objective”  (Kirkwood,  1997:24).  As  a  result,  the  degree  of  attainment  is  converted  from 
a  subjective  platform  to  a  more  objective  platform,  which  allows  for  an  easier 
measurement  of  attainment  for  a  particular  objective. 

2. 7.3.1  Types  of  Evaluation  Measure  Scales 

Evaluation  measure  scales  are  classified  as  being  either  natural  or  constructed  and 
either  direct  or  proxy.  A  natural  scale  is  a  scale  “that  is  in  general  use  with  a  common 
interpretation  by  everyone”  (Kirkwood,  1997:24).  For  example,  price  in  dollars  would  be 
a  natural  scale  for  the  cost  value  for  the  “Buy  the  Best  Truck”  decision.  A  constructed 
scale  “is  one  that  is  developed  for  a  particular  decision  problem  to  measure  the  degree  of 
attainment  of  an  objective”  (Kirkwood,  1997:24).  In  other  words,  constructed  scales  are 
used  “when  there  is  no  existing  natural  scale”  available  (Kirkwood,  1997:24).  In  the 
“Buy  the  Best  Truck”  decision,  an  example  of  a  constructed  scale  would  be  the 
categorical  levels  of  two  door,  extended  cab,  and  crew  cab  for  the  style  value. 

In  addition  to  having  a  natural  or  constructed  scale,  an  evaluation  measure  can 
also  have  either  a  direct  or  proxy  scale.  According  to  Kirkwood  (1997:24),  “a  direct 
scale  directly  measures  the  degree  of  attainment  of  an  objective,  while  a  proxy  scale 
reflects  the  degree  of  attainment  of  its  associated  objective,  but  does  not  directly  measure 
this.”  Thus,  cost  in  dollars  would  be  an  example  of  a  direct  scale  and  the  number  of 
championships  to  measure  a  football  team’s  success  would  be  an  example  of  a  proxy 
scale. 
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2. 7.3.2  Desired  Properties  of  Value  Hierarchies 


When  selecting  evaluation  measure  scales,  there  are  three  desirable  properties 
decision  makers  should  consider:  measurability,  operationality,  and  understandability 
(Keeney,  1992:1 12-116).  The  property  of  measurability  “defines  the  associated  objective 
(value)  in  more  detail  than  that  provided  by  the  objective  alone”  (Keeney,  1992: 1 13). 
Thus,  measurability  ensures  the  evaluation  measure  scale  precisely  measures  the  value 
envisioned  by  the  decision  maker.  Operationality  “express(es)  relative  preferences  for 
different  levels  of  achievement  of  an  objective  (value)  as  indicated  by  attribute  levels” 
(Keeney,  1992:1 14).  The  property  of  understandability  eliminates  ambiguity  in 
describing  and  interpreting  consequences  in  terms  of  attributes  (Keeney,  1992:116).  The 
incorporation  of  these  properties  in  value  hierarchies  aides  in  clarifying  respective 
objectives  (values)  and  facilitating  VFT  (Keeney,  1992:112). 

2.7.4  Step  4  -  Value  Function  Creation 

Once  the  evaluation  measure  scales  are  determined,  the  evaluation  measures  must 
have  the  capability  of  converting  differing  units  and  scales  into  a  common  scale,  which 
helps  with  being  able  to  score  and  compare  alternatives.  The  Single-Dimension  Value 
Function  (SDVF)  provides  this  capability  by  combining  “the  multiple  evaluation 
measures  into  a  single  measure  of  the  overall  value  of  each  evaluation  alternative” 
(Kirkwood,  1997:53).  Thus,  the  SDVF  converts  units  of  an  evaluation  measure  into 
“value  units”  by  assigning  a  value  from  0  to  1 .  The  SDVF  is  composed  of  two  axes,  the 
x-axis  and  the  y-axis.  The  x-axis  consists  of  a  set  of  points  used  to  represent  the 
evaluation  of  a  particular  measure.  The  value  for  the  measure  is  represented  on  the  y- 
axis.  When  a  decision  maker  assigns  each  point  on  the  x-axis  a  value  on  the  y-axis,  a 
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function  is  created.  As  a  result,  the  decision  maker  has  the  capability  of  placing  all 
measures  on  the  same  “unit-less”  scale  (Weir,  2003). 

There  are  two  different  types  of  value  functions:  piecewise  linear  and  exponential 
linear.  The  piecewise  linear  function  is  “made  up  of  segments  of  straight  lines  that  are 
joined  together,”  whereas  the  exponential  linear  function  “uses  a  specific  mathematical 
form”  to  help  convert  each  individual  measure’s  units  into  value  units.  For  these 
functions,  the  least  preferred  score  for  a  particular  evaluation  measure  will  have  a  value 
of  zero  while  the  most  preferred  score  will  have  a  value  of  one  (Kirkwood,  1997:61).  In 
both  cases,  value  functions  can  take  on  monotonically  increasing  or  decreasing  shapes.  A 
monotonically  increasing  function  will  have  an  increase  in  value  on  the  y-axis  as  the 
score  increases  on  the  x-axis.  Conversely,  a  monotonically  decreasing  function  will  have 
a  decrease  in  value  on  the  y-axis  as  the  score  increases  on  the  x-axis.  Examples  of 
monotonically  increasing  and  decreasing  value  functions  are  shown  in  Figures  5  and  6, 
respectively. 


Figure  5.  Monotonically  Increasing  Exponential  (left)  and  Piecewise  Linear  (right) 

Value  Functions 
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Figure  6.  Monotonically  Decreasing  Exponential  (left)  and  Piecewise  Linear  (right) 

Value  Functions 

2.7.5  Step  5  -  Value  Hierarchy  Weighting 

Once  the  value  functions  are  created,  each  value  must  be  differentiated  according  to 
its  relative  importance.  The  decision  maker  can  accomplish  this  by  assigning  weights  to 
each  value  in  the  value  hierarchy,  with  the  entire  value  hierarchy  usually  receiving  a  total 
weight  of  one.  This  concept  is  illustrated  in  the  “Buy  the  Best  Truck”  hierarchy  shown  in 
Figure  7  (Shoviak,  2001:57).  Accordingly,  each  value  in  the  hierarchy  is  assigned  a 
portion  of  the  total  weight.  This  score  or  weight  reflects  the  decision  maker’s  preference 
for  each  value.  The  weights  can  be  assigned  on  a  local  and  global  basis 
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Figure  7.  “Buy  the  Best  Truck”  Value  Hierarchy  with  Local  Weights  (Jurk,  2002:45) 


2. 7.5.1  Local  Weighting 

Local  weighting  deals  with  the  differentiation  of  values  on  the  same  tier  within  a 
single  branch  of  the  value  hierarchy.  The  sum  of  all  the  local  weights  within  a  tier  of  a 
branch  must  sum  to  one.  For  example,  Figure  8  shows  the  first-tier  values  of 
Performance,  Practicality,  and  Safety  (Jurk,  2002:44).  Since  the  weights  on  the  first-tier 
sum  to  one,  the  weights  are  considered  to  be  local.  In  a  similar  manner,  the  weights  for 
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the  branch  of  the  second-tier  values  of  Safety  (i.e.,  Off-Road  and  On-Road)  and  On  Road 
(i.e.,  ABS  and  Crash  Test  Rating)  are  considered  local  weights  since  they  sum  to  one. 


Figure  8.  “Buy  the  Best  Truck”  Local  Weights  Example  (Jurk,  2002:46) 

2. 7. 5. 2  Global  Weighting 

Once  the  local  weighting  for  all  values  and  measures  is  complete,  the  global 
weights  can  be  detennined.  The  global  weights  are  derived  from  the  local  weights  by 
“multiplying  the  local  weights  for  each  successive  tier  above  it”  (Katzer,  2002:43).  The 
global  weights  are  used  in  the  overall  calculation  of  the  value  hierarchy.  As  illustrated  in 
Figure  9,  the  global  weights  on  any  given  tier  of  the  hierarchy  must  sum  to  one. 
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Figure  9.  “Buy  the  Best  Truck”  Global  Weights  Example  (Jurk,  2002:46) 


2.7.6  Step  6  -  Alternative  Generation 

After  completing  the  hierarchy  and  weighting  the  values,  alternatives  for  the 
decision  problem  are  generated.  Normally,  the  value  hierarchy  is  used  as  a  starting  point 
for  generating  alternatives.  If  too  many  alternatives  are  generated,  the  value  functions  of 
the  hierarchy  can  serve  as  the  screening  criterion.  Thus,  if  a  measure  for  a  particular 
alternative  returns  a  score  of  “zero”  in  the  value  function,  it  may  give  motivation  to 
eliminate  the  alternative  containing  this  measure  from  the  model.  By  contrast,  when  too 
few  alternatives  are  generated,  the  hierarchy  can  identify  value  gaps.  Identifying  value 
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gaps  is  instrumental  in  modifying  the  hierarchy  in  order  for  alternatives  to  score  better  in 
critical  areas. 

2.7.7  Step  7  -  Alternative  Scoring 

After  generating  alternatives,  data  must  be  collected  and  translated  into  each 
measure’s  SDVF.  As  mentioned  in  Step  4,  the  SDVF  assigns  a  value  (or  score)  from  0  to 
1  based  on  where  the  data  is  positioned  on  the  SDVF’s  x-axis.  Since  the  x-axis  is  the 
driving  force  behind  alternative  scoring,  the  y-axis  is  often  hidden  from  stakeholders. 

This  method,  termed  “blind  scoring,”  eliminates  any  potential  scoring  bias  by  removing 
any  ties  to  the  hierarchy  weighting.  Each  measure  is  scored  one  at  a  time.  As  an  ultimate 
goal,  the  decision  maker  should  ensure  the  data  collected  for  each  measure  is  clearly 
defined,  understandable  to  everyone,  and  has  proper  documentation. 

2.7.8  Step  8  -  Deterministic  Analysis 

The  deterministic  analysis  consists  of  a  mathematical  equation  that  combines  a 
particular  alternative’s  respective  SDFVs  (Step  4),  and  associated  weights  (Step  5).  As  a 
result,  the  combination  of  data  creates  an  aggregated  score  or  weighted  sum  score  for 
each  alternative.  Thus,  the  weighted  sum  score  provides  insight  for  the  decision  maker  to 
rank  order  the  VFT  model  results. 

The  additive  value  function  is  a  mathematical  equation  used  by  many  in  decision 
analysis  for  the  rank  ordering  of  alternatives  (Shoviak,  2001 :60).  This  simplistic 
mathematical  equation  provides  the  decision  maker  with  the  means  to  conduct  detailed 
sensitivity  analysis  of  alternatives  (Kirkwood,  1997:230;  Shoviak,  2001:60).  There  are 
several  prerequisites  before  a  decision  maker  can  use  this  equation.  First,  each  evaluation 
measure  must  have  an  SDVF  with  an  assigned  weight.  Next,  the  SDVFs  must  be 
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constructed  with  the  objective  of  calculating  values  between  0  (lowest  score)  and  1 
(highest  score).  Finally,  the  combined  weights  for  a  particular  alternative  must  sum  to 
one.  Once  all  prerequisites  are  met,  the  decision  maker  can  “construct  a  strategically 
equivalent  additive  value  function”  which  takes  the  following  form  (Kirkwood, 
1997:230): 

v(x)  =  YjAivi(xi) 

i= 1 

The  value  function  is  represented  by  v(x),  where  v;(x,  )  is  the  translated  score  from  the 
alternative’s  SDVF  and  At  is  the  scaling  constant  or  associated  weight. 

2.7.9  Step  9  -  Conduct  Sensitivity  Analysis 

After  completing  the  deterministic  analysis,  the  decision  maker  can  perform 
sensitivity  analysis.  The  sensitivity  analysis  expands  on  the  detenninistic  analysis  by 
answering  the  following  question  for  the  decision  maker:  “How  would  this  decision 
change  if  another  interested  party  had  weighted  the  hierarchy  or  provided  the  data  for  the 
SDVFs?”  (Clemen  et  al,  2001 : 175;  Katzer,  2002:46).  Since  there  is  little  change  in  the 
SDVFs,  the  sensitivity  analysis  is  performed  with  the  model’s  weights.  When  the 
weights  are  used  for  sensitivity  analysis,  the  value  of  the  weight  under  consideration  is 
varied  from  0.000  to  1.000  while  the  other  dependent  weights  remain  proportionally 
constant.  Thus,  the  variation  of  relative  importance  for  the  weight  under  consideration 
can  be  explored  and  presented  on  a  breakeven  chart. 

2.7.10  Step  10  -  Recommendations  and  Conclusions 

After  completing  the  deterministic  analysis,  the  decision  maker  can  perform 
sensitivity  analysis.  The  sensitivity  analysis  expands  on  the  detenninistic  analysis  by 
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answering  the  following  question  for  the  decision  maker:  “How  would  this  decision 
change  if  another  interested  party  had  weighted  the  hierarchy  or  provided  the  data  for  the 
SDVFs?”  (Clemen  et  al,  2001 : 175;  Katzer,  2002:46).  Since  there  is  little  change  in  the 
SDVFs,  the  sensitivity  analysis  is  performed  with  the  model’s  weights.  When  the 
weights  are  used  for  sensitivity  analysis,  the  value  of  the  weight  under  consideration  is 
varied  from  0.000  to  1.000  while  the  other  dependent  weights  remain  proportionally 
constant.  Thus,  the  variation  of  relative  importance  for  the  weight  under  consideration 
can  be  explored  and  presented  on  a  breakeven  chart. 
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Chapter  3.  Methodology 


This  chapter  explains  how  the  value  model  was  developed  to  help  detennine  Air 
Force  Utility  Privatization  effectiveness.  The  chapter  details  the  first  seven  steps  of  the 
10-step  process  discussed  in  Chapter  2  and  present  the  elements  of  the  model. 

3.1  Step  1  -  Problem  Identification 

Before  the  problem  can  be  solved,  the  exact  nature  of  the  problem  or  the  overall 
fundamental  objective  must  be  defined.  For  this  thesis,  the  fundamental  objective  is  to 
determine  the  effectiveness  of  Air  Force  utility  privatization.  In  order  to  accomplish  this 
objective,  a  mathematical  model  must  be  built  to  evaluate  Air  Force  utility  privatization 
from  a  post-award  point  of  view.  Thus,  the  model  must  be  capable  of  evaluating, 
scoring,  and  ranking  all  privatized  Air  Force  utility  systems  based  on  the  values 
important  to  Air  Force  decision  makers.  Once  created,  the  model,  along  with  a  privatized 
utility  system  database  containing  measurable  data,  will  serve  as  a  decision  support 
model.  It  will  provide  Air  Force  decision  makers  with  the  oversight  and  insight  needed 
to  make  its  utility  privatization  program  successful.  The  Value-Focused  Thinking  (VFT) 
process  was  used  to  help  create  this  mathematical  model  to  solve  the  decision  problem. 

The  decision  maker  for  this  problem  is  the  Air  Force’s  Utility  Privatization 
Program  Manager  at  the  Air  Force  Civil  Engineer  Support  Agency  (AFCESA)  at  Tyndall 
Air  Force  Base,  Florida.  The  Program  Manager’s  objective  for  this  thesis  is  to  have  a 
detailed  listing  of  all  the  major  values  and  sub-values  that  should  be  considered  when 
evaluating  the  perfonnance  of  a  privatized  utility  system.  In  addition,  the  decision  maker 
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wants  to  know  how  the  values  compare  to  each  other  in  terms  of  level  of  importance. 
Furthermore,  the  decision  maker  desires  to  have  the  capability  to  identify  and  present  the 
distinct  differences  among  the  privatized  utility  systems  being  evaluated.  Thus,  the 
problem  statement  is  “Determine  the  effectiveness  of  each  privatized  Air  Force  utility 
system.”  This  problem  statement  represents  the  basis  for  the  fundamental  objective  of 
this  decision  problem. 

3.2  Step  2  -  Value  Hierarchy  Construction 

After  clearly  defining  the  problem,  the  next  step  was  constructing  the  value 
hierarchy  to  solve  it.  For  this  step,  the  fundamental  objective  was  iteratively  divided  into 
specific  values  until  they  could  be  measured.  To  begin  the  process,  the  researcher  and  a 
stakeholder  used  the  gold  standard  to  construct  a  preliminary  structure  known  as  the 
“strawman”  hierarchy.  The  purpose  of  the  “strawman”  hierarchy  is  to  assist  the  decision 
maker  in  generating  values  for  the  decision  problem.  The  Air  Force  utility  privatization 
Request  for  Proposal  (RFP)  template  document  was  used  to  generate  and  list  all  values 
relating  to  the  fundamental  objective.  Next,  an  affinity  grouping  exercise  was  used  to 
logically  detennine  the  values  and  sub-values  of  the  “strawman”  hierarchy.  The 
“strawman”  hierarchy  organized  the  values  into  hierarchical  format  starting  with  the 
overarching  value  at  the  top  and  working  down  to  measurable  values  at  the  bottom.  A 
total  of  18  values  were  grouped  into  two  headings,  which  are  Cost  and  Performance. 
Figure  10  illustrates  the  “strawman”  hierarchy  with  seven-tiers  of  values  and/or 
measures. 
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Tier  1 


Tier  2 


Tier  3 


Tier  4 


Tier 


Figure  10.  Strawman  Hierarchy 
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After  completing  the  “strawman”  hierarchy,  the  platinum  standard  was  used  during 
a  two-day  meeting  with  the  researcher,  the  proxy  decision  maker  from  AFCESA,  and 
several  subject  matter  experts  to  help  identify  overlooked  values  imperative  to  the 
hierarchy.  The  “strawman”  hierarchy  was  presented  in  order  to  prompt  the  generation  of 
values  essential  to  the  problem  statement.  Each  meeting  attendee  provided  what  he  or 
she  believed  to  be  important  considerations  for  privatized  utility  system  evaluation. 
During  the  meeting,  the  proxy  decision  maker  directed  that  the  Cost  value  be  removed 
from  the  hierarchy  because  AFCESA  is  only  concerned  with  performance  at  this  time. 
One  of  the  subject  matter  experts  supported  the  proxy  decision  maker’s  suggestion  by 
explaining  that  the  lack  of  tier  depth  in  the  Cost  value  would  create  weighting  problems 
for  the  entire  hierarchy.  After  the  removal  of  the  Cost  value,  the  remaining  values 
underneath  the  Performance  value  were  grouped  into  three  main  headings — Quality, 
Reliability,  and  Responsiveness. 

The  values  making  up  the  Quality  and  Responsiveness  branches  remained 
unaltered;  however,  one  value,  System  Efficiency,  was  removed  from  the  Reliability 
branch.  The  difficulty  in  differentiating  the  amount  of  contribution  a  privatized  utility 
provider  (contractor)  is  actually  contributing  to  overall  utility  system  efficiency  was  the 
main  reason  for  the  removal  of  this  value.  The  proxy  decision  suggested  that  further 
research  should  be  conducted  to  determine  a  reliable  method  to  account  for  a  contractor’s 
system  efficiency.  Therefore,  the  System  Efficiency  value  can  be  incorporated  into  the 
model  at  a  later  time.  Figure  1 1  illustrates  the  complete  value  hierarchy  for  Air  Force 
utility  privatization  evaluation.  The  details  of  the  hierarchy  will  be  explained  in  the 
remainder  of  this  section. 
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Tier  1 


Tier  2 


Tier  3 


Tier  4 


Tier  5 


Figure  11.  Final  Utility  Privatization  Evaluation  Value  Hierarchy 
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3.2.1  Decompose  Fundamental  Objective 


The  main  objective  of  the  Air  Force  utility  privatization  evaluation  hierarchy  is  to 
determine  the  effectiveness  of  the  Air  Force’s  privatized  utility  systems.  To  understand 
how  utility  system  privatization  effectiveness  can  be  evaluated,  the  fundamental  objective 
was  decomposed  into  three  distinct  branches:  (1)  Quality,  (2)  Reliability,  and  (3) 
Responsiveness .  The  hierarchy  illustrating  the  values  comprising  the  first  and  second-tier 
is  shown  in  Figure  12. 

3.2.2  Quality  Branch 

The  first  of  the  three  branches  in  the  first-tier  of  the  hierarchy  is  the  Quality  branch. 
Quality  for  Air  Force  privatized  utility  systems  is  obtained  when  the  contracted  utility 
provider  (contractor)  provides  a  safe,  secured,  and  environmentally  sound  utility  system. 
In  addition,  the  contractor  must  provide  the  Air  Force  with  the  capability  to  measure  the 
amount  of  utility  commodity  (i.e.,  electricity,  water,  and  natural  gas)  being  consumed  by 
its  installations.  The  second-tier  values  of  Effective  Administration,  Environmen  tal 
Stewardship,  Utility  System  Safety,  Sub-Metering  Capability,  and  Utility  System  Security 
further  define  the  Quality  branch  by  providing  more  detailed  information.  The  values 
comprising  the  first  and  second-tier  of  the  Quality  branch  are  shown  in  Figure  13. 
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Tier  1  Tier  2 


Figure  12.  Utility  Privatization  Evaluation  Hierarchy  Showing  Tiers  1  and  2 
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Figure  13.  Utility  Privatization  Evaluation  Hierarchy  for  Quality  Branch 
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3.2.2. 1  Effective  Administration 


The  first  of  the  five  values  in  the  second-tier  of  the  Quality  branch  is  Effective 
Administration,  which  is  obtained  when  a  contractor  maintains,  updates,  and  stores 
documents  essential  to  the  operation  of  an  Air  Force’s  utility  system.  The  third-tier  of 
values  Maintaining  Proper  Licenses,  Permits,  &  Certification;  Maintaining  Sendee 
Records  for  Two  Years;  and  Main  taining  and  Updating  Drawings  further  define  the 
Effective  Administration  value  by  providing  more  detailed  information. 

3.2. 2. 1.1  Maintaining  Proper  Licenses,  Permits,  &  Certifications 

Maintaining  Proper  Licenses,  Permits,  &  Certifications  are  requirements  for  all 

contractors  providing  utilities  to  Air  Force  installations.  This  value  reflects  the  Air 
Force’s  desire  to  have  all  of  its  utility  systems  in  compliance  with  federal,  state,  and  local 
regulations  and  laws.  Also,  the  contractor  is  “responsible  for  obtaining  any  new  or 
revised  pennits,  [licenses],  or  [certifications]  needed  to  operate  and  maintain  the  utility 
system”  (Department  of  the  Air  Force,  2003: 19).  To  ensure  this  compliance,  contractors 
are  required  to  ensure  their  operations  “meet  all  applicable  federal,  state,  local,  and 
installation  certification,  licensing,  and  medical  requirements  to  perfonn  all  assigned 
tasks  and  functions  as  defined  in  the  contract”  (Department  of  the  Air  Force,  2003:13). 
This  value  does  not  include  safety  requirements. 

3. 2. 2. 1.2  Maintaining  Service  Records  for  Two  Years 

Maintaining  Service  Records  for  Two  Years  is  a  requirement  for  all  contractors 
providing  utilities  to  Air  Force  installations.  This  value  reflects  the  Air  Force’s  desire  to 
retrieve  service  records  from  contractors  in  order  to  measure  their  ability  to  provide 
utility  service  to  the  installation.  The  contractor  must  ensure  that  their  operations  “record 
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all  service  request  calls,  documenting  the  time  of  the  call,  time  of  service  response,  cause 
of  request,  and  action  taken  (including  time  and  date  completed)”  (Department  of  the  Air 
Force,  2003: 17).  The  contractor  must  maintain  these  records  for  at  least  a  two-year  time 
period.  These  records  “may  be  reviewed  by  the  Administrative  Contracting  Officer  upon 
reasonable  request  and  with  reasonable  notice”  (Department  of  the  Air  Force,  2003:17). 

3. 2.2. 1.3  Maintaining  and  Updating  Drawings 

The  contractor  must  “maintain  record  drawings  for  all  existing  and  new  facilities 
installed  by  the  [contractor  within  the  service  area”  (Department  of  the  Air  Force, 
2003:13).  This  value  reflects  the  Air  Force’s  desire  to  request  these  drawings  from  the 
contractor  in  order  to  use  them  and  make  copies  for  its  own  purposes.  In  addition,  the 
contractor  must  provide  the  Air  Force  with  these  drawings  “in  the  form  of  CAD-CAM 
disks. .  .using  the  latest  release  software  compatible”  with  Air  Force  systems  (Department 
of  the  Air  Force,  2003: 13).  Furthermore,  “the  contractor  will  also  provide  information  to 
allow  for  updates  to  the  installation  Geographical  Information  System  (GIS),  as 
appropriate”  (Department  of  the  Air  Force,  2003:13). 

3. 2. 2. 2  Environmental  Stewardship 

The  second  of  the  five  values  in  the  second-tier  of  the  Quality  branch  is 
Environmental  Stewardship.  This  value  is  obtained  when  a  contractor  provides  an 
effective  contingency  plan  for  spills,  minimizes  hazardous  waste  and  materials,  and 
increases  the  installation’s  solid  waste  diversion  rate  by  recycling  appropriate  materials 
used  in  its  daily  operations.  The  third-tier  values  of  Effective  Spill  Contingency  Plan  and 
Hazardous  Material/  Waste  Minimization  and  Recycling  provide  more  detailed 
information  about  the  Environmental  Stewardship  value. 
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3.2. 2.2.1  Effective  Spill  Contingency  Plan 

The  Effective  Spill  Contingency  Plan  value  reflects  the  Air  Force’s  desire  to  have 
the  contractor’s  spill  contingency  plan  reflect  the  installation’s  spill  contingency  plan.  In 
addition,  the  contractor’s  spill  contingency  plan  should  “be  developed  in  accordance  with 
the  National  Response  Team’s  Integrated  Contingency  Plan  Guidance”  (Department  of 
the  Air  Force,  2003: 19). 

3.2. 2.2. 2  Hazardous  Material/Waste  Minimization  and  Recycling 

The  Hazardous  Material/Waste  Minimization  and  Recycling  value  reflects  the  Air 

Force’s  desire  to  have  contractors  handle  hazardous  materials  and  recycle  reusable 
materials  according  to  applicable  laws  and  regulations.  For  hazardous  materials  used  on 
the  installation,  the  contractor  is  required  to  have  appropriate  Material  Safety  Data  Sheets 
(MSDDs).  In  addition,  the  contractor  is  required  to  “maintain  a  viable  hazardous  waste 
minimization  program  that  includes  making  every  effort  to  identify  non-hazardous  or  less 
hazardous  materials  than  those  currently  in  use”  (Department  of  the  Air  Force,  2003: 19). 
As  for  recycling,  the  contractor  is  required  to  divert  all  reusable  materials  from  the 
installation’s  waste  stream  by  recycling  it  according  to  applicable  laws  and  regulations. 

3.2.2. 3  Utility  System  Safety 

The  third  of  the  five  values  in  the  second-tier  of  the  Quality  branch  is  Utility  System 
Safety.  Utility  System  Safety  is  obtained  when  a  contractor  performs  utility  system 
distribution,  construction,  and  maintenance  within  the  guidelines  of  applicable  laws  and 
regulations.  The  third-tier  values  of  Compliance  with  Utility  System  Laws/Regulations, 
Decreased  Utility  System  Mishaps,  and  Employee  Safety  Certification  provide  more 
detailed  information  about  the  Utility  System  Safety  value. 
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3. 2.2. 3.1  Compliance  with  Utility  System  Laws/Regulations 

The  Compliance  with  Utility  System  Laws/Regulations  value  reflects  the  Air 

Force’s  desire  to  have  all  its  utility  systems  in  compliance  with  federal,  state,  and  local 
safety,  fire  prevention,  and  health  codes.  The  contractor  is  required  to  adhere  to  all 
federal,  state,  and  local  safety,  fire  prevention,  and  health  codes. 

3.2.23.2  Decreased  Utility  System  Mishaps 

The  Decreased  Utility  System  Mishaps  value  reflects  the  Air  Force’s  desire  to  have 
the  least  amount  of  utility  system  mishaps  as  possible.  A  utility  system  mishap  is  defined 
as  an  event  that  causes  the  loss  of  man-hours  or  resources  due  to  poor  safety  practices. 

3.2.23.3  Employee  Safety  Certification 

The  Employee  Safety  Certification  value  reflects  the  Air  Force’s  desire  to  have 
contractors  fully  certified  in  safety  procedures.  Unlike  the  value  in  the  Effective 
Administration  branch,  the  value  focuses  on  the  employee  safety  requirements.  Thus, 
contractors  are  required  to  have  their  employees  “meet  all  applicable  federal,  state,  local, 
and  installation  [safety]  certification,  licensing,  and  medical  requirements  (i.e.,  CPR)” 
(Department  of  the  Air  Force,  2003: 13). 

3. 2.2. 4  Utility  System  Security 

The  fourth  of  the  five  values  in  the  second-tier  of  the  Quality  branch  is  Utility 
System  Security,  which  reflects  the  Air  Force’s  desire  to  have  safe  and  secured  utility 
systems.  Also  reflected  in  this  value  is  the  Air  Force’s  desire  to  have  contractors  ensure 
their  employees  do  not  present  “a  potential  threat  to  the  health,  safety,  security,  general 
well  being,  or  operational  mission  of  the  Installation  or  population”  (Department  of  the 
Air  Force,  2003:13). 
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3.2.2. 5  Sub-Metering  Capability 


The  last  of  the  five  values  in  the  second-tier  of  the  Quality  branch  is  Sub-Metering 
Capability.  This  value  reflects  the  Air  Force’s  desire  to  have  its  contractor  provide  the 
capability  to  measure  the  utility  system’s  commodity  consumption  on  the  installation 
through  the  use  of  sub-metering.  The  Air  Force  “will  use  sub-meters  for  internal 
installation  billing  purposes  and  for  commodity  management  and  energy  conservation 
purposes”  (Department  of  the  Air  Force,  2003: 1 1).  The  contractor  is  responsible  reading, 
maintaining,  and  calibrating  all  sub-meters  on  the  installation.  In  addition,  the  contractor 
is  responsible  for  installing,  reading,  maintaining,  and  calibrating  any  future  sub-meters 
as  requested  by  the  Air  Force. 

3.2.3  Reliability  Branch 

The  second  of  the  three  branches  in  the  first-tier  of  the  hierarchy  is  the  Reliability 
branch.  Reliability  for  Air  Force  privatized  utility  systems  is  obtained  when  the 
contractor  provides  continuous  and  dependable  utility  service  with  minimal  interruptions. 
The  second-tier  values  of  Completed  Renewals/Replacements  and  Decreased  Utility 


Figure  14.  Utility  Privatization  Evaluation  Hierarchy  for  Reliability  Branch 
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3.23.1  Completed  Renewals/Replacements 


The  first  of  the  two  values  in  the  second-tier  of  the  Reliability  branch  is  Completed 
Renewals/Replacements.  This  value  reflects  the  Air  Force’s  desire  to  have  “continuing 
maintenance,  repairs,  and  upgrades  that  will  permit  the  long-term  safe  and  reliable 
operation  of  utility  system[s]”  (Department  of  the  Air  Force,  2003:20).  Completed 
renewals  and  replacements  will  help  keep  Air  Force  utility  systems  in  compliance  “with 
all  requirements  and  standards  imposed  by  law  as  well  as  the  standards  typically  applied 
by  the  Contractor  to  its  other  utility  systems”  (Department  of  the  Air  Force,  2003:20). 

3.23.2  Decreased  Utility  System  Outages 

The  second  of  the  two  values  in  the  second-tier  of  the  Reliability  branch  is 
Decreased  Outages,  which  reflects  the  Air  Force’s  desire  to  minimize  unscheduled 
service  interruptions  for  its  utility  systems.  When  there  is  an  unscheduled  service 
interruption,  the  contractor  is  required  to  record  the  following  at  a  minimum:  cause  of 
interruption,  detailed  contingency  plan  of  action,  estimated  time  for  reestablishment  of 
temporary  service,  and  estimated  time  for  reestablishment  of  pennanent  service. 

3.2.4  Responsiveness  Branch 

The  last  of  the  three  branches  in  the  first-tier  of  the  hierarchy  is  the  Responsiveness 
branch.  Responsiveness  for  Air  Force  privatized  utility  systems  is  obtained  when  the 
contractor  is  able  to  timely  respond  to  service  requests  and  other  requests,  as  designated 
by  the  Air  Force  and  installation.  The  second-tier  values  of  Effective  Digging 
Permits/Line  Marking  Program,  High  Contracting  Meeting  Attendance,  Timely  Meter 
Reading,  and  Timely  Service  Response  provide  more  detailed  information  about  the 
Responsiveness  branch.  Unlike  the  previous  two  branches,  the  Responsiveness  branch 
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has  a  third  and  fourth  level  of  tiers.  The  second,  third,  and  fourth-tier  values  comprising 
the  Responsiveness  branch  are  shown  in  Figure  15. 

S.2.4.1  Effective  Digging  Permits/Line  Marking  Program 

The  first  of  the  four  values  in  the  second-tier  of  the  Responsiveness  branch  is 
Effective  Digging  Permits/Line  Marking  Program.  This  value  reflects  the  Air  Force’s 
desire  to  have  the  contractor  effectively  assist  installation  organizations  and  personnel 
with  digging  permits  and  marking  underground  utility  lines. 
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Tier  1  Tier  2  Tier  3  Tier  4 


Figure  15.  Utility  Privatization  Evaluation  Hierarchy  for  Responsiveness  Branch 
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3. 2. 4. 2  High  Contracting  Meeting  Attendance 


The  second  of  the  four  values  in  the  second-tier  of  the  Responsiveness  branch  is 
High  Contracting  Meeting  Attendance,  which  reflects  the  Air  Force’s  desire  to  have 
contractors  “available  for  meetings  as  reasonably  required  by  the  Administrative 
Contracting  Officer”  (Department  of  the  Air  Force,  2003: 18).  The  meetings  are  held  to 
discuss  matters  concerning  the  contractor’s  performance  and  needs. 

3. 2. 4. 3  Timely  Meter  Readings 

Timely  Meter  Readings  is  the  third  of  the  four  values  in  the  second-tier  of  the 
Responsiveness  branch.  The  Timely  Meter  Readings  value  reflects  the  Air  Force’s  desire 
to  have  the  contractor  read  meters  and  submit  meter-reading  reports  in  a  timely  matter. 
Timely  submission  of  meter  reading  reports  by  the  contractor  will  allow  the  Air  Force  to 
accurately  bill  installation  reimbursable  customers. 

3. 2. 4. 4  Timely  Service 

The  last  of  the  four  values  in  the  second-tier  of  the  Responsiveness  branch  is  Timely 
Sendee.  Timely  Service  is  obtained  when  the  contractor  responds  to  all  emergency, 
urgent,  and  routine  service  requests  in  a  timely  manner.  The  third-tier  values  of  Timely 
Emergency  Service  Response,  Timely  Urgent  Service  Response,  and  Timely  Routine 
Sendee  Response  provide  more  detailed  information  about  the  Timely  Service  value. 

3.2. 4.4.1  Timely  Emersency  Service  Response 

The  first  of  the  three  values  in  the  third-tier  of  the  Timely  Sendee  branch  is  Timely 
Emergency  Service  Response,  which  reflects  the  Air  Force’s  desire  to  have  the  contractor 
respond  to  emergency  conditions  in  a  timely  manner.  “An  emergency  condition  is  one 
that  is  detrimental  to  the  mission  of  the  [installation],  significantly  impacts  operational 
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effectiveness,  or  compromises  the  safety,  health,  and  life  of  personnel”  (Department  of 
the  Air  Force,  2003:16).  Therefore,  emergency  service  requests  can  include  inoperative 
airfield  lighting,  water  outages,  electrical  outages,  and  downed  natural  gas/power  lines. 
The  forth-tier  values  listed  below  provide  more  detailed  information  about  the  Timely 
Emergency  Service  Response  value. 

Adequate  24/7  Hotline  Support :  This  value  reflects  the  Air  Force’s  desire 
to  have  the  contractor  available  to  respond  to  emergency  service  requests 
24  hours  a  day,  every  day  (24/7).  The  contractor  is  required  to  have  a 
service  request  line  in  place  for  base  personnel  to  call. 

Timely  Initial  Emergency  Response :  This  value  reflects  the  Air  Force’s 
desire  to  have  the  contractor  send  “a  representative  knowledgeable  of  the 
[utility]  system  and  the  service  [interruption  procedures]  on  the  site  of  the 
emergency  within  1  hour”  (Department  of  the  Air  Force,  2003:16). 

Timely  Emergency  Crew  Response :  This  value  reflects  the  Air  Force’s 
desire  to  have  the  contractor  send  “repair  crews  appropriately  trained  to 
eliminate  the  condition”  on  the  site  of  the  emergency  within  two  hours. 

Timely  Remedied  Emergency  Response:  This  value  reflects  the  Air 
Force’s  desire  to  have  the  contractor  remedy  or  downgrade  the  emergency 
condition  in  a  timely  manner.  The  contractor  is  required  to  remedy  or 
downgrade  all  emergencies  “to  a  non-emergency  status  within  24  hours” 
(Department  of  the  Air  Force,  2003: 16).  “For  regulated  utilities,  the 
service  and  its  restoration  in  times  of  outage  for  emergency  service 
requests  shall  be  at  least  equivalent  to  the  service  provided  to  other  similar 
customers”  (Department  of  the  Air  Force,  2003: 16). 

Timely  Response  to  Exercises/Contingencies:  This  value  reflects  the  Air 
Force’s  desire  to  have  the  contractor  “respond  to  installation  emergency 
and  crisis  situations  and  exercises  for  emergency  and  crisis  situations  that 
require  utility  support”  (Department  of  the  Air  Force,  2003:18).  The 
contractor  is  required  to  respond  to  exercises  and  contingencies  “with 
qualified  personnel  and  equipment  as  soon  as  possible  after  notification 
during  nonnal  duty”  (Department  of  the  Air  Force,  2003: 18). 

Effective  Emergency  FACMAN  Coordination:  This  value  reflects  the  Air 
Force’s  desire  to  have  the  contractor  coordinate  with  installation  facility 
managers  (FACMAN)  if  the  emergency  request  affects  their  facility.  If 
the  emergency  request  affects  building  operations,  the  contractor  is 
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required  to  coordinate  all  work  with  the  person  responsible  for  the 
building  or  facility. 

3.2. 4.4. 2  Timely  Urgent  Service  Response 

Timely  Urgent  Service  Response  is  the  second  of  the  three  values  in  the  third-tier  of 
the  Timely  Service  branch.  This  value  reflects  the  Air  Force’s  desire  to  have  the 
contractor  respond  to  urgent  conditions  in  a  timely  manner.  “An  urgent  condition  is  not 
an  emergency  but  significantly  hinders  performance  of  installation  activities  and  requires 
elimination  of  potential  fire,  health,  and  safety  hazards”  (Department  of  the  Air  Force, 
2003:16).  Therefore,  urgent  service  requests  can  include  downgraded  emergency 
responses;  environmental  controls;  and  special  requests  and  events.  The  fourth-tier 
values  listed  below  provide  more  detailed  information  about  the  Timely  Urgent  Service 
Response  value. 

Timely  Initial  Urgent  Response:  This  value  reflects  the  Air  Force’s  desire 
to  have  the  contractor  send  “a  representative  knowledgeable  of  the  [utility] 
system  and  the  service  [interruption  procedures]  on  the  site  of  the  [urgent] 
request  within  24  hours”  (Department  of  the  Air  Force,  2003: 16). 

Timely  Remedied  Urgent  Response:  This  value  reflects  the  Air  Force’s 
desire  to  have  the  contractor  remedy  the  urgent  condition  in  a  timely 
manner.  The  contractor  is  required  to  remedy  the  urgent  condition  within 
five  calendar  days.  “For  regulated  utilities,  the  service  and  its  restoration 
in  times  of  outage  for  urgent  service  requests  shall  be  at  least  equivalent  to 
the  service  provided  to  other  similar  customers”  (Department  of  the  Air 
Force,  2003:16). 

Effective  Urgent  FACMAN  Coordination:  This  value  reflects  the  Air 
Force’s  desire  to  have  the  contractor  coordinate  with  FACMAN  if  the 
urgent  request  affects  their  facility.  If  the  urgent  request  affects  building 
operations,  the  contractor  is  required  to  coordinate  all  work  with  the 
person  responsible  for  the  building  or  facility. 

3.2. 4.4. 3  Timely  Routine  Service  Response 

The  last  of  the  three  values  in  the  third-tier  of  the  Timely  Service  branch  is  Timely 
Routine  Service  Response,  which  reflects  the  Air  Force’s  desire  to  have  the  contractor 
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respond  to  routine  conditions  in  a  timely  manner.  “A  routine  service  request  is  one  that 

does  not  pose  an  immediate  threat  to  public  health,  safety,  or  property,  or  to  a  mission  or 

operation  conducted  at  the  installation”  (Department  of  the  Air  Force,  2003:19). 

Therefore,  routine  service  requests  can  include,  “but  are  not  necessarily  limited  to 

requests  for  new  or  relocated  service  connections”  (Department  of  the  Air  Force, 

2003:19).  The  fourth-tier  values  listed  below  provide  more  detailed  information  about  the 

Timely  Routine  Service  Response  value. 

Timely  Initial  Routine  Response :  This  value  reflects  the  Air  Force’s  desire  to 
have  the  contractor  respond  to  routine  service  requests  within  a  timely  manner. 
“The  contractor  is  not  required  to  respond  to  the  Installation’s  routine  service 
requests  outside  normal  duty  hours.  The  contractor  may  respond  to  routine 
service  requests  outside  of  normal  duty  hours  at  its  option  and  with  appropriate 
coordination”  (Department  of  the  Air  Force,  2003: 17).  However,  the  contractor  is 
required  to  initially  respond  to  any  routine  service  request  within  five  calendar 
days.  “For  regulated  utilities,  the  service  and  its  restoration  in  times  of  outage  for 
routine  service  requests  shall  be  at  least  equivalent  to  the  service  provided  to  other 
similar  customers”  (Department  of  the  Air  Force,  2003: 17). 

Timely  Remedied  Routine  Response :  This  value  reflects  the  Air  Force’s  desire  to 
have  the  contractor  remedy  the  routine  condition  in  a  timely  manner.  The 
contractor  is  required  to  remedy  the  routine  condition  within  30  calendar  days. 
“For  regulated  utilities,  the  service  and  its  restoration  in  times  of  outage  for 
routine  service  requests  shall  be  at  least  equivalent  to  the  service  provided  to  other 
similar  customers”  (Department  of  the  Air  Force,  2003: 17). 

Effective  Two-Week  Coordination :  This  value  reflects  the  Air  Force’s  desire  to 
have  the  contractor  coordinate  at  least  two  weeks  prior  to  commencing  work  for  a 
routine  service  request.  The  contractor  is  required  to  coordinate  “with  the 
Contracting  Officer’s  Representative  at  least  two  weeks  prior  to  commencing” 
routine  work,  “such  as  the  scheduled  repair,  replacement,  or  removal  of  system 
components  that  require  service  interruption”  (Department  of  the  Air  Force, 
2003:19). 

Effective  Routine  FACMAN  Coordination :  This  value  reflects  the  Air  Force’s 
desire  to  have  the  contractor  coordinate  with  FACMAN  if  the  routine  request 
affects  their  facility.  If  the  routine  request  affects  building  operations,  the 
contractor  is  required  to  coordinate  all  work  with  the  person  responsible  for  the 
building  or  facility. 
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3.3  Step  3  -  Develop  Evaluation  Measures 


Developing  measures  is  the  next  step  in  constructing  the  hierarchy.  As  discussed  in 
Chapter  2,  evaluation  measures  are  used  to  capture  the  degree  of  attainment  for  values  in 
a  hierarchy.  In  addition,  evaluation  measures  allow  the  decision  maker  to  convert  the 
degree  of  attainment  from  a  subjective  platform  to  a  more  objective  platfonn,  which 
allows  for  easier  measurement  of  attainment  for  values.  The  initial  measures  were 
developed  with  the  aid  of  a  subject  matter  expert;  during  the  process,  it  was  kept  in  mind 
that  the  evaluation  measures  should  be  easily  understood  and  have  data  readily  available. 
This  approach  made  the  task  of  developing  evaluation  measures  simple. 

After  developing  the  suggested  measures,  they  were  presented  at  a  two-day  meeting 
for  validation.  The  proxy  decision  maker  agreed  with  a  majority  of  the  measures,  except 
for  the  measures  under  the  Environmental  Stewardship,  Utility  System  Safety,  and  Timely 
Response  to  Exercises/Contingencies  values.  The  proxy  decision  maker  stressed  that  the 
degree  of  attainment  for  these  measures  did  not  directly  reflect  the  impact  to  the 
installation’s  mission.  In  order  to  reflect  the  desires  of  the  decision  maker,  the  measures 
in  question  were  later  verified  and  revamped  with  subject  matter  experts  in  the  fields  of 
safety,  environmental  compliance,  and  inspection  programs. 

As  discussed  in  Chapter  2,  the  desirable  evaluation  measure  properties  of 
measurability,  operability,  and  understandability  were  incorporated  during  the  measure 
development  process.  As  a  result,  a  total  of  47  evaluation  measures  were  developed  for 
the  hierarchy.  The  evaluation  measures  were  grouped  into  two  measure  types — 
natural/direct  and  constructed/direct.  The  natural/direct  measure  type  uses  natural 
attributes  to  measure  quantities  that  directly  reflect  a  value.  Natural/direct  measures 
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generally  have  known  and  common  measure  units.  An  example  of  a  natural/direct 
measure  developed  for  this  hierarchy  is  the  Average  Number  of  Days  to  Update  measure, 
which  uses  the  measure  unit  of  days.  The  constructed/direct  measure  type  is  used  when 
no  natural  attribute  exists  to  measure  quantities  that  directly  reflect  a  value. 
Constructed/direct  measures  are  generally  based  on  combinations  of  information  that 
pertain  to  the  value.  As  a  result,  a  subjective  qualitative  rating  is  created  to  help  capture 
the  degree  of  attainment  for  constructed/direct  measures.  An  example  of  a 
constructed/direct  measure  developed  for  this  hierarchy  is  the  Rating  from 
Exercises/Contingencies  measure,  which  uses  the  measure  unit  of  rating.  The  measures 
and  measure  definitions  for  each  first-tier  value  are  displayed  in  Tables  1  and  2, 
respectively.  The  remaining  measures  and  definitions  are  contained  in  Appendices  A  and 
B,  respectively. 


Table  1.  Examples  of  Measures 


Branch 

Value 

Associated 

Measure 

Measure 

Unit 

Lower 

Bound 

Upper 

Bound 

Quality 

Maintaining 
Proper 
Licenses, 
Permits,  & 
Certification 

Percentage  of 
Up-to-Date 
Licenses, 
Permits,  and 
Certifications 

Percentage 

Natural/Direct 

0% 

100% 

Reliability 

Completed 

Renewals/ 

Replacements 

Percentage  of 
Items  Actually 
Renewed  or 
Replaced 

Percentage 

Natural/Direct 

0% 

100% 

Responsiveness 

Effective 

Digging 

Permits/Line 

Marking 

Program 

Number  of 
Utility  Line 
Hits 

Hits 

Natural/Direct 

0 

20 
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Table  2.  Example  of  Definitions 


Measure 

Definition 

Percentage  of  Up-to-Date 
Licenses,  Permits,  and 
Certifications 

The  percentage  of  all  licenses,  permits,  and  certifications  the  contractor  is 
keeping  up-to-date.  The  percentage  is  determined  by  the  ratio  of  actual 
number  of  licenses,  permits,  and  certifications  held  by  the  contractor  to  the 
number  of  up-to-date  licenses,  permits,  and  certifications  held  by  the 
contractor. 

Percentage  of  Items 
Actually  Renewed  or 
Replaced 

The  percentage  of  items  actually  renewed  or  replaced  by  the  contractor.  A 
renewal/replacement  list  is  normally  created  by  the  contractor  with  the 
consent  of  the  Air  Force  a  year  in  advance  to  help  schedule  continuing 
maintenance,  repairs,  and  upgrades  for  the  installation  utility  system.  The 
list  is  normally  executed  a  year  later.  The  percentage  is  determined  by  the 
ratio  of  the  actual  number  of  completed  renewals  and  replacements  for  a 
one-year  period  to  the  number  of  scheduled  renewals  and  replacements  for 
that  same  one-year  period.  : 

Number  of  Utility  Line 
Hits 

The  number  of  utility  line  hits.  The  number  of  line  hits  is  an  indicator  of 
the  effectiveness  of  the  contractor's  digging  permit  and  line  marking 
program. 

3.4  Step  4  -  Create  Value  Functions 

The  next  step  in  the  VFT  process  is  to  define  a  value  function  for  each  evaluation 
measure  developed  in  Step  3.  As  mentioned  in  Chapter  2,  Single  Dimension  Value 
Functions  (SDVF)  are  developed  to  convert  differing  units  and  scales  of  evaluation 
measures  for  a  hierarchy  into  a  common  scale,  which  can  help  score  and  rank 
alternatives.  The  value  for  each  evaluation  measure  is  set  to  a  range  of  0.000  (least 
preferred)  to  1.000  (most  preferred)  along  the  y-axis.  The  set  of  points  used  to  represent 
the  evaluation  of  a  particular  measure  is  set  to  a  range  of  the  decision  maker’s  most 
desirable  preference  to  least  desirable  preference  (or  vice-versa)  along  the  x-axis. 
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The  SDVFs  for  this  hierarchy  were  developed  by  proxy  decision  makers  using  a 
direct  assessment  technique.  The  direct  assessment  technique  involves  the  proxy 
decision  makers  adjusting  the  curves  and  scales  for  each  SDVF.  For  this  step,  a 
document  was  created  and  electronically  mailed  to  the  proxy  decision  makers  to  facilitate 
the  development  of  the  SDVFs.  The  document  consists  of  several  EXCEL  spreadsheets, 
which  allow  the  proxy  decision  makers  to  assign  preference  on  the  x-axis  and  adjust  the 
shape  of  each  measure’s  SDVF.  After  several  iterations,  47  SDVFs  were  developed 
under  three  value  function  categories — discrete,  monotonically  increasing  exponential, 
and  monotonically  decreasing  exponential.  Examples  of  these  SDVF  are  discussed 
below.  The  remaining  SDVFs  are  displayed  in  Appendix  C. 

Discrete  SDVFs  are  normally  used  when  evaluation  measures  have  a  small  number 
of  possible  scoring  levels.  An  example  of  a  discrete  SDFV  developed  for  this  hierarchy 
is  the  Average  Number  of  Days  to  Update  function  shown  in  Figure  16,  which  measures 
how  long  it  takes  a  contractor  to  update  drawings.  The  most  preferred  score  is  for  a 
contractor  that  is  able  to  update  drawings  under  60  days,  and  it  receives  a  value  of  1.000. 
The  second  most  preferred  score  is  a  contractor  that  was  able  to  update  drawings  between 
60  to  65  days,  thereby  receiving  a  value  of  0.838.  The  third  most  preferred  score  is  for  a 
contractor  that  is  able  to  update  drawings  between  66  to  70  days,  and  it  receives  a  value 
of  0.420.  The  least  preferred  score  is  for  a  contractor  that  updates  the  drawings  in  more 
than  75  days,  thereby  receiving  a  value  of  0.000.  The  second  least  preferred  score  is  for  a 
contractor  that  is  only  able  to  update  drawings  between  71  to  75  days,  and  it  receives  a 
value  of  0.180. 
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Label  Value 


Under  60  days 
60  -  65  days 
66-  70  days 
71  -  75  days 
Over  75  days 


1.000 

0.838 

0.420 

0.180 

0.000 


Figure  16.  Average  Number  of  Days  to  Update  SDVF 


Monotonically  increasing  exponential  SDFVs  are  normally  used  when  there  is  a 
preference  for  increases  to  the  score  on  the  x-axis.  For  every  increase  in  score  on  the  x- 
axis,  the  value  on  the  y-axis  will  increase  exponentially.  An  example  of  a  monotonically 
increasing  exponential  SDVF  developed  for  this  hierarchy  is  the  Percentage  of  Up-to- 
Date  Licenses,  Permits,  and  Certifications  function  shown  in  Figure  17  measures  the 
contractor’s  percentage  of  up-to-date  licenses,  permits,  and  certifications.  The  most 
preferred  score  is  for  a  contractor  that  is  able  to  keep  all  licenses,  permits,  and 
certifications  up-to-date,  thereby  receiving  a  value  of  1.000.  The  least  preferred  score  is 
for  a  contractor  that  is  unable  to  keep  any  licenses,  permits,  and  certifications  up-to-date, 
and  it  receives  a  value  of  0.000.  As  illustrated  in  Figure  17,  the  contractor’s  value  on  the 
y-axis  exponentially  increases  for  every  increase  in  percentage  on  the  x-axis. 
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Figure  17.  Percentage  of  Up-to-Date  Licenses,  Pennits,  Permits,  and  Certifications 

SDVF 


Monotonically  decreasing  exponential  SDVFs  are  normally  used  when  there  is  a 
preference  for  decreases  to  the  score  on  the  x-axis.  For  every  decrease  in  score  on  the  x- 
axis,  the  value  on  the  y-axis  will  decrease  exponentially.  An  example  of  a  monotonically 
decreasing  exponential  SDVF  developed  for  this  hierarchy  is  the  Number  of  Minor 
Findings  for  Spill  Contingency  Plan  function  shown  in  Figure  18,  measures  the  number 
of  minor  findings  the  contractor  received  for  its  spill  contingency  plan.  The  most 
preferred  score  is  for  a  contractor  that  received  no  minor  findings,  thereby  receiving  a 
value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  that  received  up  to  20  minor 
findings,  and  it  receives  a  value  of  0.000.  As  illustrated  in  Figure  18,  the  contractor’s 
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value  on  the  y-axis  exponentially  increases  for  every  decrease  in  percentage  on  the  x- 


axis. 


Figure  18.  Number  of  Minor  Findings  for  Spill  Contingency  Plan  SDVF 

3.5  Step  5  -  Weight  the  Value  Hierarchy 

As  explained  in  Chapter  2,  each  value  in  the  hierarchy  must  be  differentiated 
according  to  how  the  decision  maker  perceives  its  relative  importance.  This  step  of  the 
VFT  process  is  accomplished  by  having  the  decision  maker  assign  weights  to  each  value 
in  the  hierarchy.  For  this  step,  a  document  was  created  and  electronically  mailed  to  the 
decision  maker  to  facilitate  weighting  the  values.  This  document  consisted  of  a  detailed 
drawing  of  the  value  hierarchy  with  text  boxes  for  each  value.  These  text  boxes  allowed 
the  decision  maker  to  use  the  direct  weighting  technique  to  adjust  the  suggested  local 
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weights  to  reflect  his  preference  for  each  value.  Once  all  the  local  weights  were 
assigned,  the  document  was  sent  back  to  the  researcher  for  calculation  of  global  weights. 
The  resulting  local  and  global  weights  for  the  hierarchy  are  described  in  the  remainder  of 
this  section. 

3.5.1  Assignment  of  Local  Weights 

Using  the  direct  weighting  technique  as  previously  discussed,  the  decision  maker 
first  assigned  local  weights  to  the  values  comprising  the  first-tier  of  the  hierarchy.  The 
following  values  of  Quality,  Reliability,  and  Responsiveness  were  assigned  weights  of 
0.300,  0.350,  and  0.350  respectively.  The  weights  for  the  first-tier  are  shown  in  Figure 
19.  The  following  sections  will  discuss  the  weights  assigned  to  the  values  comprising  the 
branches  for  each  of  the  first- tier  values. 


Figure  19.  Local  Weights  for  the  First-Tier  Values 
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3. 5. 1.1  Local  Weights  for  Quality  Branch 


The  direct  weighting  technique  was  used  to  assign  local  weights  to  every  value 
comprising  the  Quality  branch.  First,  local  weights  were  assigned  to  the  second-tier 
values  of  the  Quality  branch.  The  highest  weight  of  0.350  was  assigned  to  the  Utility 
System  Security  value.  Due  to  the  United  States’  ongoing  war  with  terrorism  and  the  Air 
Force’s  desire  to  keep  their  utility  systems  secured  from  terrorist  attacks,  the  decision 
maker  felt  that  the  Utility  System  Security  value  should  receive  the  most  emphasis.  The 
remaining  values,  Utility  System  Safety,  Effective  Administration,  Environmental 
Stewardship,  and  Sub-Metering  Capability,  received  weights  of  0.250,  0.150,  0.150,  and 
0.100,  respectively.  The  weights  for  the  remaining  values  and  measures  in  each  tier  of 
the  Quality  branch  are  shown  in  Figures  20  and  21.  Note:  the  Quality  branch  is  divided 
between  these  two  figures  because  of  its  size. 
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Figure  20.  Local  Weights  for  Quality  Branch 
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Tier  2  Tier  3 


0.500 


Figure  2 1 .  Local  Weights  for  Quality  Br 
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#  of  RAC  4—  Negligible  Violations 

0.050 


#  of  RAC  3--  Moderate  Violations 

0.150 


#  of  RAC  2-  Critical  Violations 

0.300 


#  of  RAC  1  —  Catastrophic  Violations 

0.500 


%  of  Employees  Completing  all  Requirements 

1.000 


(Continued) 


3. 5. 1.2  Local  Weights  for  Reliability  Branch 


For  the  Reliability  branch,  the  second-tier  values  of  Completed 
Renewals/Replacements  and  Decreased  Utility  System  Outages  received  weights  of 
0.400  and  0.600,  respectively.  The  Decreased  Utility  System  Outages  received  the 
highest  weight  because  of  the  Air  Force’s  desire  to  have  the  contractor  lower  the  numbers 
of  utility  system  outages  an  Air  Force  installation  experiences  each  year.  The  weights  for 
every  value  and  measure  in  the  Reliability  branch  are  shown  in  Figure  22. 


Figure  22.  Local  Weights  for  Reliability  Branch 


3. 5. 1.3  Local  Weights  for  Responsiveness  Branch 

For  the  Responsiveness  branch,  the  second -tier  values  of  Effective  Digging 
Permits/Line  Marking  Program,  High  Contracting  Meeting  Attendance,  Timely  Meter 
Reading,  and  Timely  Sendee  Response  received  weights  of  0. 100,  0. 100,  0. 100,  and 
0.700,  respectively.  The  Timely  Service  Response  value  received  the  highest  weight 
because  of  the  decision  maker’s  desire  to  have  contractor’s  respond  to  service  calls  in  a 
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timely  manner.  The  remaining  values  in  the  second-tier  of  the  Responsiveness  branch 
were  of  equal  importance;  therefore,  these  values  received  the  same  weight.  The  weights 


for  every  value  and  measure  in  the  Responsiveness  branch  are  shown  in  Figures  23  and 
24.  Note:  The  Reliability  branch  is  divided  between  the  two  figures  because  of  its  size. 
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Tier  2 


Tier  3 


Tier  4 


Tier  5 


Effective  Digging  Femit/Line  Marking  Program 

0.100 


Timely  Meter  Reading 

0.100 


High  Contracting  Meeting  Attendance 

0.100 


#  of  Utility  Line  Efts 

1.000 


%of  Late  Meter  Reading) 
1.000 


%of  IVfeetings  Attended 

1.000 


Timely  Remeded  Emergency  Response 

0200 


Timely  Response  to  Exercises/Conti  ngendes 

0200 


%of  Q»al  Met  for  Timely  Emergency  OewResponse 

1.000 


%of  Qial  Met  for  Ti  mely  Remedied  Emergency  Response 

1.000 


#  of  Uisatisfactory  Ratings  from  Exerdses/Contingendes 

0.200 


#  of  Marginal  Ratings  thnn  Exercises/Conti  ngencies 

0.200 


#  rf  Satisfactory  Rati  ng  from  Exercises/Conti  i 

0.200 


#  rf  Excel  lent  Rating)  from  Exercises/Conti  ngendes 

0.200  y 


Figure  23.  Local  Weights  for  Responsiveness  Branch 
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Tier  3  Tier  4  Tier  5 


Figure  24.  Local  Weights  for  Responsiveness  Branch  (Continued) 

3.5.2  Assignment  of  Global  Weights 

The  global  weights  for  the  hierarchy  are  obtained  by  multiplying  the  local  weight 
of  the  value  (or  measure)  being  looked  at  by  the  local  weight  of  each  value  in  the  branch 
above  the  value  (or  measure)  until  the  fundamental  objective  is  reached.  For  example,  as 
illustrated  in  Figure  25,  the  global  weight  of  the  Percentage  of  Items  Actually  Replaced 
measure  (0. 140)  is  calculated  by  multiplying  the  local  weight  of  Percentage  of  Items 
Actually  Replaced  ( 1 .000)  by  the  local  weight  of  the  Completed  Renewals/Replacements 
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value  (0.400)  and  by  the  local  weight  of  the  Reliability  value  (0.350).  The  calculations  of 
global  weights  for  the  remaining  values  are  in  Appendix  D.  The  rank  order  of  the 
measures  by  global  weight  is  shown  in  Appendix  E. 


Figure  25.  Global  Weight  Calculation  Example 


3.6  Summary 

This  chapter  covered  how  the  values  and  measures  comprising  the  utility 
privatization  evaluation  hierarchy  were  created.  The  characteristics  valued  by  the 
decision  maker  and  stakeholders  associated  with  the  decision  problem  were  used  to  guide 
the  development  of  the  evaluation  model.  Alternative  generation  and  scoring,  along  with 
deterministic  and  sensitivity  analyses  (i.e.,  Steps  through  9)  are  conducted  in  Chapter  4. 
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Chapter  4.  Results  and  Analysis 


This  chapter  presents  Steps  6  through  9  of  the  Value-Focused  Thinking  (VFT) 
process.  In  Step  6,  notional  data  is  generated  to  create  alternatives  for  comparison  in  the 
evaluation  model.  In  Step  7,  the  evaluation  model  scores  the  alternatives  created  in  Step 
6.  Deterministic  analysis  of  the  alternatives  is  conducted  in  Step  8;  this  provides  the 
decision  maker  a  rank  ordering  of  the  alternatives  and  insight  into  the  performance  of  the 
alternatives.  In  Step  9,  sensitivity  analysis  is  performed  on  the  local  weights  of  first-tier 
values  to  investigate  their  impact  on  the  alternative  rankings. 

4.1  Step  6  -  Alternative  Generation 

Once  weighting  the  hierarchy  is  complete,  the  next  step  in  the  VFT  process  is  to 
identify  alternatives  that  can  be  evaluated  with  the  hierarchy.  For  this  research  problem. 
Step  6  of  the  standard  VFT  process  was  modified  slightly.  Since  the  Air  Force  has  only 
recently  made  the  decision  to  privatize  non-essential  utility  systems,  few  systems  were 
available  for  evaluation.  Consequently,  there  was  not  enough  data  to  adequately  reflect 
the  perfonnance  of  these  utility  systems.  Therefore,  notational  data  was  generated  to 
represent  a  variety  of  privatized  utility  systems  operating  in  the  near  future. 

The  notional  data  created  for  this  research  represents  a  variety  of  privatized  Air 
Force  utility  systems  operating  in  the  near  future.  An  EXCEL  spreadsheet  was  used  to 
randomly  generate  over  500  possible  alternatives  representing  a  privatized  utility  system. 
From  the  generated  notional  data,  eight  alternatives  were  selected  that  best  represent 
privatized  utility  systems  with  overall  scores  in  the  following  categories:  above  average 
(90%  -  100%),  average  (70%  to  80%),  and  below  average  (40%  to  50%).  These 
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categories  were  established  to  help  demonstrate  how  the  model  could  be  used  to  analyze 
and  compare  actual  utility  system  data.  The  notional  data  used  to  create  the  alternatives 
is  provided  in  Appendix  F. 

4.2  Step  7  -  Alternative  Scoring 

Before  any  analysis  of  the  alternatives  could  be  accomplished,  the  alternatives  were 
evaluated  (i.e.,  scored).  For  this  step,  the  evaluation  measures,  value  functions,  and 
weights  were  used  to  create  an  aggregate  value  for  each  simulated  alternative.  Using  the 
“blind  scoring”  technique,  the  notational  data  for  each  alternative  was  collected  and 
scored  using  the  single  dimension  value  functions  (SDVFs)  for  the  measures.  The 
scoring  results  are  presented  in  Table  3. 


Table  3.  Scoring  Results  for  Alternatives 


Alternative 

Category 

Score 

Base  7 

Above  Average 

0.960 

Base  5 

Average 

0.795 

Base  2 

Average 

0.734 

Base  8 

Average 

0.717 

Base  4 

Below  Average 

0.553 

Base  1 

Below  Average 

0.533 

Base  6 

Below  Average 

0.467 

Base  3 

Below  Average 

0.441 
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4.3  Step  8  -  Deterministic  Analysis 


The  deterministic  analysis  was  used  to  examine  the  initial  results  of  the  evaluation 
model  and  provide  insight  to  the  decision  maker  regarding  the  ranking  of  the  privatized 
utility  systems.  For  this  step,  the  additive  value  function  was  used  to  incorporate 
hierarchy  weights  with  the  alternative  scores.  As  a  result,  a  weighted  sum  value  was 
created  that  can  be  used  to  rank  order  the  alternatives.  A  bar  graph  of  the  detenninistic 
analysis  results  for  the  notional  data  is  provided  in  Figure  26. 


Alternative 

Score 

Base  7 

0.960 

Base  5 

0.795 

Base  2 

0.734 

Base  8 

0.717 

Base  4 

0.553 

Base  1 

0.533 

Base  6 

0.467 

Base  3 

0.441 

Responsiveness  Reliability 


Quality 


Figure  26.  Alternative  Score  Ranking  and  Detenninistic  Analysis  Results 


The  bar  graph  demonstrates  how  the  model  can  rank  order  the  performance  of  Air 
Force  privatized  utility  systems.  Equally  important,  this  graph  provides  insight  into 
which  privatized  utility  systems  are  perfonning  well  or  need  improvement.  Moreover, 
the  graph  presents  each  alternative’s  value  in  a  cumulative  bar.  The  bars  representing 
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each  alternative’s  score  is  partitioned  to  show  the  amount  of  value  each  first-tier  value. 
Accordingly,  the  partitions  facilitate  the  deterministic  analysis  by  providing  the  decision 
maker  with  a  clear  and  easy  way  to  analyze  how  well  each  privatized  utility  system  is 
performing  in  the  areas  of  Quality,  Reliability,  and  Responsiveness.  A  decision  maker 
can  use  the  graph  to  guide  the  investigation  in  detennining  why  certain  privatized  utility 
systems  are  performing  below  standards.  For  instance,  the  graph  in  Figure  26  illustrates 
that  Base  8’s  low  score  in  the  Reliability  branch  is  the  reason  for  its  ranking  below  Base 
2.  This  can  prompt  a  decision  maker  to  further  analyze  the  scores  Base  8  received  for  the 
measures  within  the  Reliability  branch  in  order  to  target  the  exact  areas  that  need 
improvement. 

4.4  Step  9  -  Sensitivity  Analysis 

Sensitivity  analysis  was  conducted  on  the  local  weights  of  each  of  the  first-tier 
values  to  detennine  how  changes  in  the  value  weights  can  impact  the  ranking  of  the 
alternatives.  In  other  words,  sensitivity  analysis  is  used  to  provide  the  decision  maker 
with  insight  into  how  the  alternative  rankings  might  change  if  another  stakeholder  (senior 
Air  Force  leadership)  had  weighted  the  hierarchy.  Also,  the  slope  of  the  line  for  each 
alternative  generated  by  the  sensitivity  analysis  can  be  used  to  determine  a  privatized 
utility  system’s  performance  strength  in  achieving  a  particular  area  of  evaluation.  This 
can  be  determined  by  calculating  the  difference  between  the  alternative’s  slopes  when  the 
value  receives  weights  of  0.000  and  1 .000,  respectively.  For  example,  a  steep  upward 
sloping  line  with  a  slope  calculation  of  +33%  is  a  strong  indicator  that  a  privatized  utility 
system  is  most  likely  performing  well  in  a  particular  first-tier  value  compared  to  the  other 
first-tier  values.  However,  a  steep  downward  sloping  line  with  a  slope  calculation  of  - 
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37%  is  normally  a  strong  indicator  that  a  privatized  utility  system  is  not  performing  well 
in  a  particular  first- tier  value  compared  to  the  other  first- tier  values.  Ultimately,  the 
insight  provided  from  the  sensitivity  analysis  provides  the  decision  maker  with  a  more 
informed  decision.  Sensitivity  analysis  was  accomplished  by  varying  the  weight  of  one 
of  the  first-tier  values  while  adjusting  the  weights  of  the  remaining  values  to  maintain 
their  original  proportionate  weighting 

4.4.1  Quality  Branch  Sensitivity  Analysis 

The  first-tier  value  of  Quality  was  assigned  an  initial  local  weight  of  0.300. 
Sensitivity  analysis  was  accomplished  by  varying  the  local  weight  from  0.000  to  1.000 
while  proportionally  holding  the  weights  of  the  other  first-tier  values  steady.  The 
sensitivity  analysis  in  Figure  27  illustrates  that  Base  7  is  the  most  dominant  alternative 
because  it  is  insensitive  to  changes  in  weight.  This  means  the  ranking  for  Base  7  would 
remain  unchanged  regardless  of  the  weight  assigned  to  the  Quality  branch.  However,  as 
the  local  weight  is  increased  from  0.000  to  1.000,  the  overall  ranking  for  the  remaining 
bases  change.  For  example,  Base  6  is  the  least  preferred  alternative  when  the  local 
weight  for  the  Quality  branch  is  at  0.000.  Nevertheless,  as  the  weight  steadily  increases 
to  0.200,  Base  3  becomes  the  least  preferred  alternative.  Once  the  weight  reaches  0.400, 
Base  8  outranks  Base  2,  becoming  the  third  most  preferred  alternative.  As  the  weight 
reaches  0.650,  the  ranking  for  several  alternatives  change  for  a  final  time.  Base  2  moves 
from  being  the  third  most  preferred  alternative  to  the  second  most  preferred  alternative. 
Also,  Base  6,  which  was  originally  the  least  preferred  alternative  at  weight  0.000, 
becomes  the  fifth  most  preferred  alternative.  Furthermore,  Base  4  falls  from  the  fifth 
most  preferred  alternative  at  weight  0.000  to  the  second  least  preferred  alternative. 
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Figure  27.  Quality  Branch  Sensitivity  Analysis 


The  results  for  the  slope  calculations  used  to  detennine  each  alternative’s 
performance  strength  in  the  Quality  first-tier  value  are  displayed  in  Table  4.  Although 
Base  7  is  recognized  as  the  most  dominant  alternative,  its  slightly  downward  sloping  line 
with  a  calculation  of  -13%  suggests  that  it  is  not  performing  as  well  the  other  two  values 
in  the  first-tier.  The  downward  sloping  lines  for  Bases  5,  2,  1,4,  and  3,  with  slope 
calculations  of  -16%,  -10%,  -12%,  -19%,  and  -16%,  respectively,  suggest  that  their 
privatized  utility  systems  have  a  lower  level  of  performance  in  this  first-tier  value  as  well. 
This  variance  should  prompt  further  investigation  into  the  individual  scores  comprising 
the  Quality  branch  for  each  of  these  low  performing  privatized  utility  systems.  In 
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contrast,  the  slightly  upward  sloping  lines  representing  Base  8  and  6  with  slope 
calculations  of  9%  and  7%,  respectively  suggest  that  their  privatized  utility  systems  are 
performing  better  in  the  area  of  Quality  compared  to  their  performance  in  the  other  two 
areas  of  evaluation. 


Table  4.  Slope  Calculations  for  Quality  Value 


Alternative 

0% 

Quality  Value 
Weight 
Applied 

100% 

Quality  Value 
Weight 
Applied 

Slope 

Base  1 

52% 

40% 

-12% 

Base  2 

73% 

63% 

-10% 

Base  3 

43% 

27% 

-16% 

Base  4 

56% 

37% 

-19% 

Base  5 

82% 

66% 

-16% 

Base  6 

39% 

46% 

7% 

Base  7 

100% 

87% 

-13% 

Base  8 

65% 

73% 

9% 

4.4.2  Reliability  Sensitivity  Analysis 

The  first-tier  value  of  Reliability  was  assigned  an  initial  local  weight  of  0.350.  To 
perform  sensitivity  analysis,  the  local  weight  was  varied  from  0.000  to  1.000  while 
proportionally  holding  the  weights  of  the  other  first-tier  values  steady.  As  illustrated  in 
Figure  28,  Base  7  is  still  recognized  as  the  most  dominant  alternative  for  every  variation 
of  weight  in  for  the  first-tier  value  of  Reliability.  However,  as  the  local  weight  is 
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increased  from  0.000  to  1.000,  the  overall  ranking  for  the  remaining  bases  change.  For 
example,  Base  4  is  the  second  least  preferred  alternative  when  the  local  weight  for  the 
Reliability  branch  is  at  0.000.  Conversely,  as  the  weight  steadily  increases  to  0.030,  Base 
4  becomes  the  third  least  preferred  alternative.  Once  the  weight  reaches  0.150,  Base  5 
outranks  Base  8,  becoming  the  second  most  preferred  alternative.  As  the  weight  reaches 
0.220,  Base  4’s  ranking  moved  it  up  to  the  fifth  most  preferred  alternative.  Once  the 
weight  reaches  0.300,  Base  8’s  ranking  decreases  it  from  the  third  most  preferred 
alternative  to  the  fourth  most  preferred  alternative.  Finally,  Base  8’s  ranking  falls  to  the 
fifth  most  preferred  alternative  as  the  weight  is  increased  to  0.800. 


Figure  28.  Reliability  Branch  Sensitivity  Analysis 
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The  results  for  the  slope  calculations  used  to  detennine  each  alternative’s 
performance  strength  in  the  Reliability  first-tier  value  are  displayed  in  Table  5.  The 
downward  sloping  lines  for  Bases  8,  6,  and  3,  with  slope  calculations  of  -15%,  -4%,  and  - 
1%,  respectively,  suggest  that  their  privatized  utility  systems  have  a  lower  level  of 
performance  in  the  first-tier  value  of  Reliability  compared  to  their  performance  in  the 
other  first-tier  values.  This  variance  should  prompt  investigation  into  the  individual 
scores  comprising  the  Reliability  branch  for  each  low  perfonning  privatized  utility 
system.  The  steep  upward  sloping  lines  for  Bases  5  and  4  with  slope  calculations  of  24, 
suggest  that  their  privatized  utility  systems  are  performing  extremely  well  in  the  area  of 
Reliability  compared  to  their  performance  in  the  other  first-tier  values.  The  slightly 
upward  sloping  lines  for  Bases  1,  2,  and  7,  all  with  slope  calculations  of  9%,  are 
indicators  that  their  level  of  performance  in  the  area  of  Reliability  is  slightly  better 
compared  to  their  performance  in  the  other  two  areas  of  evaluation. 


Table  5.  Slope  Calculations  for  Reliability  Value 


Alternative 

0% 

Reliability 
Value  Weight 
Applied 

100% 
Reliability 
Value  Weight 
Applied 

Slope 

Base  1 

44% 

53% 

9% 

Base  2 

65% 

75% 

9% 

Base  3 

37% 

37% 

-1% 

Base  4 

41% 

65% 

24% 

Base  5 

66% 

90% 

24% 

Base  6 

42% 

38% 

-4% 

Base  7 

91% 

100% 

9% 

Base  8 

73% 

58% 

-15% 
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4.4.3  Responsiveness  Sensitivity  Analysis 


The  first-tier  value  of  Responsiveness  was  assigned  an  initial  local  weight  of  0.350. 
To  perform  sensitivity  analysis,  the  local  weight  was  varied  from  0.000  to  1.000  while 
proportionally  holding  the  weights  of  the  other  first-tier  values  steady.  As  illustrated  in 
Figure  29,  Base  7  is  still  recognized  as  the  most  dominant  alternative.  However,  as  the 
local  weight  is  increased  from  0.000  to  1.000,  the  overall  ranking  for  the  remaining  bases 
change.  For  example,  Base  3  is  the  second  least  preferred  alternative  when  the  local 
weight  for  the  Responsiveness  branch  is  at  0.500.  Once  the  weight  reaches  0.150,  Base  5 
outranks  Base  8,  becoming  the  second  most  preferred  alternative.  As  the  weight  reaches 
0.220,  Base  4’s  ranking  moves  it  up  to  the  fifth  most  preferred  alternative.  Once  the 
weight  reaches  0.300,  Base  8’s  ranking  decreases  from  the  third  most  preferred 
alternative  to  the  fourth  most  preferred  alternative.  Finally,  Base  8’s  ranking  falls  to  the 
fifth  most  preferred  alternative  as  the  weight  is  increased  to  0.800. 
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Figure  29.  Responsiveness  Branch  Sensitivity  Analysis 


The  results  for  the  slope  calculations  used  to  detennine  each  alternative’s 
performance  strength  in  the  Responsiveness  first-tier  value  are  displayed  in  Table  6. 

Since  there  are  no  downward  sloping  lines,  this  is  a  good  indicator  that  all  utility  systems 
are  perfonning  adequately  well  in  the  area  of  Responsiveness  compared  to  their 
performance  in  the  other  first-tier  values.  However,  the  steep  upward  sloping  lines  for 
Bases  8  and  3,  with  slope  calculations  of  27%  and  41%,  respectively,  are  indicators  that 
their  utility  systems  are  perfonning  extremely  well  in  the  area  of  Responsiveness 
compared  to  the  other  first- tier  values.  The  slightly  upward  sloping  lines  for  the  other 
bases,  with  slope  calculations  ranging  from  8%  to  27%,  are  indicators  that  their  level  of 
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performance  in  the  area  of  Responsiveness  is  slightly  better  compared  to  their 
performance  in  the  other  two  areas  of  evaluation. 


Table  6.  Slope  Calculations  for  Responsiveness  Value 


Alternative 

0% 

Responsiveness 

Value  Weight 
Applied 

100% 

Responsiveness 
Value  Weight 
Applied 

Slope 

Base  1 

38% 

65% 

27% 

Base  2 

62% 

81% 

20% 

Base  3 

25% 

66% 

41% 

Base  4 

42% 

62% 

19% 

Base  5 

72% 

81% 

8% 

Base  6 

33% 

56% 

22% 

Base  7 

91% 

100% 

9% 

Base  8 

57% 

84% 

27% 
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Chapter  5.  Findings  and  Conclusions 


Chapter  5  provides  a  brief  review  of  this  research  effort  while  answering  the 
research  questions  that  were  initially  put  forth  in  Chapter  1 .  It  then  explains  how  the 
evaluation  model  created  in  this  research  can  be  used  by  the  Air  Force  to  improve  its 
utility  privatization  process.  Also,  the  evaluation  model’s  strengths  and  limitations  are 
discussed.  The  chapter  concludes  by  presenting  recommendations  for  future  research  in 
this  area. 

5.1  Review 

This  is  the  first  documented  use  of  Value-Focused  Thinking  (VFT)  to  assist 
organizations  and  governments  in  evaluating  the  performance  of  their  privatized  utility 
systems.  The  resulting  model  can  be  used  by  various  organizations  at  the  federal  and 
state  levels  of  government.  To  achieve  the  objective  of  creating  a  valid  mathematical 
model  to  evaluate  the  perfonnance  of  Air  Force  utility  privatization,  the  research 
answered  the  following  investigative  questions  addressed  in  Chapter  1 : 

1.  Given  that  quantitative  data  (performance)  will  be  collected,  what  is  a 
suitable  method  to  evaluate  and  measure  the  overall  effectiveness  of  a  utility 
system?  In  order  to  accomplish  this,  the  “hard”  quantitative  and  the  “soft” 
qualitative  factors  of  utility  privatization  must  be  balanced. 

Finding.  The  VFT  methodology  was  identified  as  a  suitable  approach  for  solving  a 
complex  decision  problem  such  as  utility  system  evaluation.  The  research  effort  applied 
the  VFT  methodology  to  produce  a  multi-objective  decision  analysis  evaluation  model 
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for  the  decision  maker.  The  model  allows  a  decision  maker  to  quantitatively  evaluate 
qualitative  factors  of  a  decision  problem. 

2.  What  are  the  major  factors  and  sub-factors  that  should  be  considered  when 
evaluating  the  performance  of  Air  Force  utility  privatization? 

Finding.  Step  2  of  the  VFT  methodology,  Value  Hierarchy  Construction,  was  used 
to  identify  the  major  factors  and  sub-factors  for  evaluating  the  perfonnance  of  Air  Force 
privatized  utility  systems.  This  was  accomplished  by  having  the  decision  maker  define 
“what  is  important  to  them  in  terms  of  utility  system  evaluation.”  In  order  to  detennine 
what  is  important,  the  decision  maker  had  to  generate  and  define  the  major  factors  and 
sub-factors,  known  as  values,  which  are  fundamental  to  the  decision  problem.  The  values 
were  structured  in  a  hierarchical  fashion  to  facilitate  the  evaluation  process  by  helping  the 
decision  maker  and  stakeholders  visualize  how  these  values  impact  the  perfonnance  of 
utility  systems. 

3.  How  do  the  major  factors  and  sub-factors  compare  to  each  other  in  terms 
of  importance? 

Finding.  Step  5  of  the  VFT  methodology,  Value  Hierarchy  Weighting,  was  used 
to  illustrate  how  the  values  compared  to  each  other  in  tenns  of  importance.  To  account 
for  the  varying  degrees  of  importance,  the  decision  maker  assigned  a  weight  to  each 
value.  The  decision  maker  was  instructed  to  distribute  a  portion  of  100  points  to  each 
value  within  a  tier  of  each  branch  of  the  hierarchy.  The  portion  of  the  points  each  value 
received  serves  an  indicator  of  relative  importance. 
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5.2  Model  Strengths 


The  evaluation  model  developed  in  this  research  was  created  using  a  combination 
of  the  “gold  and  platinum  standards.”  In  other  words,  written  guidance  from  the 
Department  of  Defense  and  the  stated  objectives  of  the  Air  Force’s  utility  privatization 
program  were  used.  The  model  captured  28  values  and  47  measures  relating  to  the  Air 
Force  utility  privatization  program’s  objective  of  improving  the  overall  quality, 
reliability,  and  responsiveness  of  its  utility  systems.  Sensitivity  analysis  on  the  weights 
assigned  to  each  value  explored  how  variations  in  weight  can  impact  the  overall  final 
ranking  of  utility  systems  being  evaluated.  Furthermore,  results  from  sensitivity  analysis 
can  help  identify  potential  problems  of  a  particular  utility  system.  In  others  words,  a 
utility  system’s  slope  during  sensitivity  analysis  can  serve  as  an  indicator  of  how  well  the 
utility  system  is  performing  in  a  particular  area  of  evaluation. 

Another  strength  of  this  evaluation  model  is  the  fact  that  it  remains  general  enough 
for  implementation,  with  few  adjustments,  at  all  military  installations  and  with  any  utility 
system.  This  is  based  on  the  fact  that  the  values  captured  in  the  model  can  be  linked  to 
general  Department  of  Defense  utility  privatization  guidance.  Other  military  departments 
will  undoubtedly  make  changes  to  the  model  to  fit  their  needs;  however,  the  model  will 
still  provide  a  defendable,  objective,  and  repeatable  process  for  evaluating  the 
performance  of  utility  systems. 

5.3  Model  Limitations 

The  evaluation  model  lacks  a  value  and  measure  to  evaluate  how  well  a  utility 
system  is  saving  energy.  Since  it  is  very  difficult  to  determine  the  amount  of  energy 
savings  contributed  by  a  privatized  utility  provider  and  an  installation’s  energy  program, 
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this  should  be  considered  as  a  separate  research  effort.  Also,  the  traditional  method  of 
weighting  the  hierarchy  in  the  VFT  process  was  modified  slightly  at  the  request  of  the 
decision  maker,  weights  reflecting  the  level  of  importance  for  each  value  were  created  by 
the  researcher  and  various  subject  matter  experts.  These  weights  were  later  adjusted  by 
the  decision  maker.  Finally,  the  research  did  not  evaluate  real  world  alternatives. 

5.4  Conclusions 

The  research  has  demonstrated  that  VFT  can  be  used  to  determine  the  effectiveness 
of  Air  Force  utility  privatization.  An  evaluation  model  was  developed  to  help  the  Air 
Force  evaluate  the  performance  of  its  utility  systems.  In  addition,  this  study  provides  the 
Air  Force  with  an  effective  decision  analysis  tool  which  provides  insight  into  the 
performance  of  its  privatized  utility  systems. 

5.5  Recommendations  for  Future  Work 

Follow  up  research  could  be  conducted  to  create  a  measure  for  the  “Energy 
Savings”  value.  Also,  to  truly  reflect  the  Air  Force’s  level  of  importance  for  each  value, 
senior  leadership  should  weight  the  model.  Finally,  the  model  should  be  further  validated 
using  real  world  data.  This  will  provide  the  Air  Force  with  insight  on  the  current  state  of 
its  privatized  utility  systems. 
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Appendix  A.  Value  Hierarchy  Measures 


Table  7.  Summary  of  Measures  for  the  Quality  Branch 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure  Type 

Lower 

Bound 

Upper 

Bound 

Third 

Maintaining 
Proper  Licenses, 
Permits,  & 
Certification 

Percentage  of 
Up-to-Date 
Licenses, 
Permits,  and 
Certifications 

Percentage 

Natural/Direct 

0% 

100% 

Third 

Maintaining 
Service  Records 
for  2  Y ears 

Are  Records  for 
the  Past  2  Y ears 
Properly 
Maintained? 

Yes  or  No 

Constructed/Direct 

No 

Yes 

Third 

Maintaining  and 
Updating 
Drawings 

Average 
Number  of  Days 
to  Update 

Days 

Natural/Direct 

Under  60 
Days 

Over  75 
Days 

Third 

Effective  Spill 
Contingency  Plan 

Number  of 
Positive 
Findings  for 
Spill 

Contingency 

Plan 

Findings 

Constructed/Direct 

0 

20 

Third 

Effective  Spill 
Contingency  Plan 

Number  of 
Minor  Findings 
for  Spill 
Contingency 
Plan 

Findings 

Constructed/Direct 

20 

0 

Third 

Effective  Spill 
Contingency 
Plan 

Number  of 
Major  Findings 
for  Spill 
Contingency 
Plan 

Findings 

Constructed/Direct 

20 

0 

Third 

Effective  Spill 
Contingency 
Plan 

Number  of 
Significant 
Findings  for 
Spill 

Contingency 

Plan 

Findings 

Constructed/Direct 

20 

0 

Third 

Hazmat/Hazwaste 
Minimization  and 
Recycling 

Percentage  of 
Liquid  Waste 
Diverted  from 
Landfills 

Percentage 

Natural/Direct 

0% 

100% 

Third 

Hazardous 
Material/Waste 
Minimization  and 
Recycling 

Percentage  of 
Solid  Waste 
Diverted  from 
Landfills 

Percentage 

Constructed/Direct 

0% 

100% 
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Table  7.  Summary  of  Measures  for  the  Quality  Branch  (Continued) 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure  Type 

Lower 

Bound 

Upper 

Bound 

Third 

Hazardous 
Material/ Waste 
Minimization  and 
Recycling 

Percentage  of 
Solid  Waste 
Diverted  from 
Landfills 

Percentage 

Constructed/Direct 

0% 

100% 

Third 

Hazardous 
Material/ Waste 
Minimization  and 
Recycling 

Number  of 
Positive 
Findings  for 
Recycling 
Program 

Findings 

Constructed/Direct 

0 

20 

Third 

Hazardous 
Material/ Waste 
Minimization  and 
Recycling 

Number  of 
Minor  Findings 
for  Recycling 
Program 

Findings 

Constructed/Direct 

20 

0 

Third 

Hazardous 
Material/ Waste 
Minimization  and 
Recycling 

Number  of 
Major  Findings 
for  Recycling 
Program 

Findings 

Constructed/Direct 

20 

0 

Third 

Hazardous 
Material/ Waste 
Minimization  and 
Recycling 

Number  of 
Significant 
Findings  for 
Recycling 
Program 

Findings 

Constructed/Direct 

20 

0 

Third 

Decreased 
Utility  System 
Mishaps 

Number  of  Lost 
Man-Hours  Due 
to  Utility 
System 
Mishaps 

Man- 

Hours 

Natural/Direct 

0 

150 

Third 

Decreased 
Utility  System 
Mishaps 

Number  of 
Utility  System 
Mishaps 

Mishaps 

Natural/Direct 

0 

15 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  1  - 
Catastrophic 
Violations 

Violations 

Constructed/Direct 

0 

10 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  2  - 
Critical 
Violations 

Violations 

Constructed/Direct 

0 

10 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  3  - 
Moderate 
Violations 

Violations 

Constructed/Direct 

0 

10 
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Table  7.  Summary  of  Measures  for  the  Quality  Branch  (Continued) 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure  Type 

Lower 

Bound 

Upper 

Bound 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  4  - 
Negligible 
Violations 

Violations 

Constructed/Direct 

0 

10 

Third 

Safety/Employee 

Certification 

Percentage  of 
Employees 
Completing  all 
Requirements 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Sub-Metering 

Capability 

Percentage  of 
Meters 

Calibrated  from 
Random 
Sample 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Sub-Metering 

Capability 

Percentage  of 
Total  Facilities 
Metered 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Utility  System 
Security 

Number  of 
Employees 
Identified  as 
Potential 
Threats 

Employees 

Natural/Direct 

None 

More 
than  1 

Second 

Utility  System 
Security 

Are  all 
Employee 
Security 
Clearances  Up- 
to-Date? 

Yes  or  No 

Constructed/Direct 

No 

Yes 
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Table  8.  Summary  of  Measures  for  the  Reliability  Branch 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure 

Type 

Lower 

Bound 

Upper 

Bound 

Second 

Completed 

Renewals/ 

Replacements 

Percentage  of 
Items  Actually 
Replaced 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Decreased 
Utility  System 
Outages 

Percentage  of 
Critical  Outages 
Caused  by 
System 
Management 

Percentage 

Natural/Direct 

100% 

0% 

Second 

Decreased 

Outages 

Percentage  of 
Non-Critical 
Outages  Caused 
by  System 
Management 

Percentage 

Natural/Direct 

100% 

0% 

Table  9.  Summary  of  Measures  for  the  Responsiveness  Branch 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure  Type 

Lower 

Bound 

Upper 

Bound 

Second 

Effective 

Digging 

Permits/Line 

Marking 

Program 

Number  of 
Utility  Line  Hits 

Hits 

Natural/Direct 

0 

20 

Second 

High 

Contracting 

Meeting 

Attendance 

Percentage  of 
Meetings 
Attended 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Timely  Meter 
Readings 

Percentage  of 
Late  Meter 
Readings 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Adequate  24/7 
Hotline 

Is  There  an 
Adequate  24/7 
Hotline? 

Yes  or  No 

Natural/Direct 

No 

Yes 

Fourth 

Timely  Initial 
Emergency 
Response 

Percentage  of 
Goal  Met  for 
Timely  Initial 
Emergency 
Response 

Percentage 

Natural/Direct 

0% 

100% 
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Table  9.  Summary  of  Measures  for  the  Responsiveness  Branch  (Continued) 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure  Type 

Lower 

Bound 

Upper 

Bound 

Fourth 

Timely 
Emergency 
Crew  Response 

Percentage  of 
Goal  for  Timely 
Emergency  Crew 
Response 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Timely 

Remedied 

Emergency 

Response 

Percentage  of 
Goal  Met  for 
Timely 
Remedied 
Emergency 
Response 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Timely 
Response  to 
Exercises/ 
Contingencies 

Number  of 
Outstanding 
Ratings  for 
Exercises/ 
Contingencies 

ORI  or 
Exercise 
Rating 

Constructed/Direct 

0 

20 

Fourth 

Timely 
Response  to 
Exercises/ 
Contingencies 

Number  of 
Excellent 
Ratings  for 
Exercises/ 
Contingencies 

ORI  or 
Exercise 
Rating 

Constructed/Direct 

0 

20 

Fourth 

Timely 
Response  to 
Exercises/ 
Contingencies 

Number  of 
Satisfactory 
Ratings  for 
Exercises/ 
Contingencies 

ORI  or 
Exercise 
Rating 

Constructed/Direct 

0 

20 

Fourth 

Timely 
Response  to 
Exercises/ 
Contingencies 

Number  of 
Marginal 
Ratings  for 
Exercises/ 
Contingencies 

ORI  or 
Exercise 
Rating 

Constructed/Direct 

20 

0 

Fourth 

Timely 
Response  to 
Exercises/ 
Contingencies 

Number  of 
Unsatisfactory 
Ratings  for 
Exercises/ 
Contingencies 

ORI  or 
Exercise 
Rating 

Constructed/Direct 

20 

0 

Fourth 

Effective 

FACMAN 

Emergency 

Coordination 

Percentage  of 
Goal  Met  for 
Effective 
FACMAN 
Emergency 
Coordination 

Percentage 

Natural/Direct 

0% 

100% 
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Table  9.  Summary  of  Measures  for  the  Responsiveness  Branch  (Continued) 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure  Type 

Lower 

Bound 

Upper 

Bound 

Fourth 

Timely  Initial 
Urgent 
Response 

Percentage  of 
Goal  Met  for 
Timely  Initial 
Urgent  Response 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Timely 

Remedied 

Urgent 

Response 

Percentage  of 
Goal  Met  for 
Timely 
Remedied 
Urgent  Response 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Effective 

FACMAN 

Urgent 

Coordination 

Percentage  of 
Goal  Met  for 
Effective 
FACMAN 
Urgent 
Coordination 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Timely  Initial 
Routine 
Response 

Percentage  of 
Goal  Met  for 
Timely  Initial 
Routine 
Response 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Timely 

Remedied 

Routine 

Response 

Percentage  of 
Goal  Met  for 
Timely 
Remedied 
Routine 
Response 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Effective 

2-Week 

Coordination 

Percentage  of 
Goal  Met  for 
Effective 
2-Week 
Coordination 

Percentage 

Natural/Direct 

0% 

100% 

Fourth 

Effective 

FACMAN 

Routine 

Coordination 

Percentage  of 
Goal  Met  for 
Effective 
FACMAN 
Routine 
Coordination 

Percentage 

Natural/Direct 

0% 

100% 
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Table  9.  Summary  of  Measures  for  the  Responsiveness  Branch  (Continued) 


Tier 

Value 

Associated 

Measure 

Measure 

Unit 

Measure  Type 

Lower 

Bound 

Upper 

Bound 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  1  - 
Catastrophic 
Violations 

Violations 

Constructed/Direct 

0 

10 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  2  — 
Critical 
Violations 

Violations 

Constructed/Direct 

0 

10 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  3  - 
Moderate 
Violations 

Violations 

Constructed/Direct 

0 

10 

Third 

Compliance  with 
Utility  System 
Laws/Regulations 

Number  of 
RAC  4  - 
Negligible 
Violations 

Violations 

Constructed/Direct 

0 

10 

Third 

Safety/Employee 

Certification 

Percentage  of 
Employees 
Completing  all 
Requirements 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Sub-Metering 

Capability 

Percentage  of 
Meters 

Calibrated  from 
Random  Sample 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Sub-Metering 

Capability 

Percentage  of 
Total  Facilities 
Metered 

Percentage 

Natural/Direct 

0% 

100% 

Second 

Utility  System 
Security 

Number  of 
Employees 
Identified  as 
Potential 
Threats 

Employees 

Natural/Direct 

None 

More 
than  1 

Second 

Utility  System 
Security 

Are  all 
Employee 
Security 
Clearances  Up- 
to-Date? 

Yes  or  No 

Constructed/Direct 

No 

Yes 
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Appendix  B.  Value  Hierarchy  Definitions 


Table  10.  Definitions  of  Quality  Measures 


Measure 

Definition 

Percentage  of  Up-to-Date 
Licenses,  Permits,  and 
Certifications 

The  percentage  of  all  licenses,  permits,  and  certifications  the  contractor  is 
keeping  up-to-date.  The  percentage  is  determined  by  the  ratio  of  actual  number 
of  licenses,  permits,  and  certifications  held  by  the  contractor  to  the  number  of 
up-to-date  licenses,  permits,  and  certifications  held  by  the  contractor. 

Are  Service  Records  for 
the  Past  2  Y ears  Properly 
Maintained? 

Whether  the  contractor  is  maintaining  all  service  call  request  and  safety  records 
for  at  least  a  two-year  period.  A  random  inspection  of  the  contractor's  records 
is  used  to  determine  whether  the  service  records  are  maintained  properly.  The 
random  inspection  inspects  for  several  service  request  calls  requirements:  time 
of  the  service  call,  time  of  response  to  service  call,  cause  of  service  call,  and 
action  taken.  If  an  inspection  of  one  record  does  not  contain  the  required 
information,  then  the  contractor's  records  are  not  considered  to  be  properly 
maintained.  A  sufficient  service  call  request  sample  size  for  the  random 
inspection  is  based  on  the  number  of  service  call  requests  the  installation 
received  for  a  year  time  period. 

Average  Number  of  Days 
to  Update 

The  average  number  of  days  the  contractor  would  take  to  update  drawings. 

The  average  number  of  days  is  determined  by  calculating  the  number  of  days 
between  the  completions  of  a  service  call  request  or  construction  project  to  the 
time  the  contractor  provides  the  Air  Force  with  the  updated  drawing.  Also,  the 
average  number  of  days  to  update  is  determined  by  summing  the  total  number 
of  days  for  all  drawing  updates  and  dividing  it  by  the  number  of  drawing 
updates. 

Number  of  Positive 
Findings  for  Spill 
Contingency  Plan 

The  degree  of  effectiveness  of  the  contractor's  spill  contingency  plan.  The 
ECAMP  finding  serves  as  an  indicator  of  effectiveness  for  the  contractor's  spill 
contingency  plan.  Positive  findings  are  an  observed  condition,  or  management 
practice  in  which  the  contractor  has  met  or  exceeded  the  compliance 
requirements. 

Number  of  Minor  Findings 
for  Spill  Contingency  Plan 

The  degree  of  effectiveness  of  the  contractor's  spill  contingency  plan.  The 
ECAMP  finding  serves  as  an  indicator  of  effectiveness  for  the  contractor's  spill 
contingency  plan.  Minor  findings  are  observed  administrative  or  procedural 
conditions  that  are  out  of  compliance  with  Department  of  Defense  or  Air  Force 
Instructions  at  any  level. 

Number  of  Major  Findings 
for  Spill  Contingency  Plan 

The  degree  of  effectiveness  of  the  contractor's  spill  contingency  plan.  The 
ECAMP  finding  serves  as  an  indicator  of  effectiveness  for  the  contractor's  spill 
contingency  plan.  Major  findings  are  observed  conditions  that  must  be 
corrected  in  order  to  avoid  future  threats  to  human  health,  safety,  the 
environment,  or  the  installation.  These  findings  are  normally  out  of 
compliance  with  federal,  state,  or  local  laws. 

Number  of  Significant 
Findings  for  Spill 
Contingency  Plan 

The  degree  of  effectiveness  of  the  contractor's  spill  contingency  plan.  The 
ECAMP  finding  serves  as  an  indicator  of  effectiveness  for  the  contractor's  spill 
contingency  plan.  Significant  findings  are  observed  conditions  that  pose  or 
have  the  likelihood  of  posing  an  immediate  and  direct  threat  to  human  health, 
safety,  the  environment,  or  the  installation's  mission. 
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Table  10.  Definitions  of  Quality  Measures  (Continued) 


Measure 

Definition 

Percentage  of  Liquid 
Waste  Diverted  from 
Landfills 

The  percentage  of  liquid  waste  the  contractor  diverted  from  landfills.  The 
contractor  is  required  to  report  liquid  waste  data  to  the  installation's 
environmental  representative.  The  percentage  is  determined  by  the  ratio  of  the 
total  liquid  waste  diverted  by  the  contractor  to  the  total  liquid  waste  generated  by 
the  contractor. 

Percentage  of  Solid  Waste 
Diverted  from  Landfills 

The  percentage  of  solid  waste  the  contractor  diverted  from  landfills.  The 
contractor  is  required  to  report  solid  waste  data  to  the  installation's  environmental 
representative.  The  percentage  is  determined  by  the  ratio  of  the  total  solid  waste 
diverted  by  the  contractor  to  the  total  solid  waste  generated  by  the  contractor. 

Number  of  Positive 
Findings  for  Recycling 
Program 

The  degree  of  contractor’s  contribution  to  the  installation's  recycling  program. 

The  ECAMP  finding  serves  as  an  indicator  of  contractor's  contribution  to  the 
installation's  recycling  program.  Positive  findings  are  an  observed  condition,  or 
management  practice  in  which  the  contractor  has  met  or  exceeded  the  compliance 
requirements. 

Number  of  Minor  Findings 
for  Recycling  Program 

The  degree  of  contractor’s  contribution  to  the  installation's  recycling  program. 

The  ECAMP  finding  serves  as  an  indicator  of  contractor's  contribution  to  the 
installation's  recycling  program.  Minor  findings  are  observed  administrative  or 
procedural  conditions  that  are  out  of  compliance  with  Department  of  Defense  or 

Air  Force  Instructions  at  any  level. 

Number  of  Major  Findings 
for  Recycling  Program 

The  degree  of  contractor’s  contribution  to  the  installation's  recycling  program. 

The  ECAMP  finding  serves  as  an  indicator  of  contractor's  contribution  to  the 
installation's  recycling  program.  Major  findings  are  observed  conditions  that 
must  be  corrected  in  order  to  avoid  future  threats  to  human  health,  safety,  the 
environment,  or  the  installation.  These  findings  are  normally  out  of  compliance 
with  federal,  state,  or  local  laws. 

Number  of  Significant 
Findings  for  Recycling 
Program 

The  degree  of  contractor’s  contribution  to  the  installation's  recycling  program. 

The  ECAMP  finding  serves  as  an  indicator  of  contractor's  contribution  to  the 
installation's  recycling  program.  Significant  findings  are  observed  conditions  that 
pose  or  have  the  likelihood  of  posing  an  immediate  and  direct  threat  to  human 
health,  safety,  the  environment,  or  the  installation's  mission. 

Recycling  ECAMP 
Finding 

The  degree  of  effectiveness  of  the  contractor's  recycling  program.  The  ECAMP 
finding  serves  as  an  indicator  of  effectiveness  for  the  contractor's  hazardous 
material/waste  minimization  and  recycling  program. 

Number  of  Lost  Man- 
Hours  Due  to  Utility 
System  Mishaps 

The  number  of  lost  man-hours  due  to  utility  system  mishaps  caused  by  the 
contractor's  safety  practices. 

Number  of  Utility  System 
Mishaps 

The  number  of  lost  man-hours  due  to  utility  system  mishaps  caused  by  the 
contractor's  safety  practices. 

Number  of  RAC  1  — 
Catastrophic  Violations 

The  number  of  RAC  1,  catastrophic  violations  caused  by  the  contractor  by  the 
contractor's  safety  practices.  Catastrophic  violations  are  electrical  safety 
violations  that  would  result  in  mission  failure,  death,  or  loss  of  system.  This  data 
can  be  retrieved  from  the  installation's  safety  office. 
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Table  10.  Definitions  of  Quality  Measures  (Continued) 


Measure 

Definition 

Number  of  RAC  2  - 
Critical  Violations 

The  number  of  RAC  2,  critical  violations  caused  by  the  contractor's  safety 
practices.  Critical  violations  are  electrical  safety  violations  that  would  result  in  a 
major  mission  degradation,  injury,  minor  occupational  illness,  or  damage.  This 
data  can  be  retrieved  from  the  installation's  safety  office. 

Number  of  RAC  3  — 
Moderate  Violations 

The  number  of  RAC  3,  moderate  violations  caused  by  the  contractor's  caused  by 
the  contractor's  safety  practices.  Moderate  violations  are  electrical  safety 
violations  that  would  result  in  a  minor  mission  degradation,  injury,  minor 
occupational  illness,  or  damage.  This  data  can  be  retrieved  from  the  installation's 
safety  office. 

Number  of  RAC  4  — 
Negligible  Violations 

The  number  of  RAC  4,  negligible  violations  caused  by  the  contractor's  safety 
practices.  Negligible  violations  are  electrical  safety  violations  that  would  result 
in  less  than  minor  mission  degradation,  injury,  occupational  illness,  or  system 
damage.  This  data  can  be  retrieved  from  the  installation's  safety  office. 

Percentage  of  Employees 
Completing  all  Safety 
Certification  Requirements 

The  percentage  of  contractor  employees  completing  all  safety  certification 
requirements.  The  percentage  is  determined  by  the  ration  of  the  actual  number  of 
contractor  employees  whom  completed  all  safety  certification  requirements  to  the 
number  of  contractor  employees  required  to  complete  safety  certification 
requirements. 

Percentage  of  Meters 
Calibrated  from  Random 
Sample 

The  percentage  of  meters  calibrated  by  the  contractor.  The  percentage  is 
determined  by  taking  a  sufficient  meter  sample  size  (based  on  the  number  of 
meters  on  the  installations)  and  inspecting  their  calibration  due  date.  If  the  date  is 
past  due,  then  the  meter  is  considered  to  be  uncalibrated.  Also,  the  percentage  is 
determined  by  dividing  the  number  of  calibrated  meters  in  the  sample  size  by  the 
sample  size. 

Percentage  of  Total 
Facilities  Metered 

The  percentage  of  total  facilities  on  the  installation  metered  by  the  contractor. 

The  percentage  is  determined  by  the  ratio  of  the  actual  number  of  facilities  on  the 
installation  metered  by  the  contractor  to  the  number  of  facilities  on  the 
installation. 

Number  of  Employees 
Identified  as  Potential 
Threats 

The  number  of  contractor  employees  identified  as  potential  threats.  This  data  can 
be  retrieved  from  the  installation's  security  forces  organization. 

Are  all  Employee  Security 
Clearances  Up-to-Date? 

Whether  the  contractor's  employees  have  required  up  to  date  security  clearances. 

If  an  inspection  of  one  employee's  record  reveals  that  the  employee's  security 
clearance  is  not  up  to  date,  then  the  contractor's  employees  are  considered  to  not 
have  up  to  date  security  clearances.  A  sufficient  employee  record  sample  size  for 
the  random  inspection  is  based  on  the  number  of  employee  records  maintained  by 
the  contractor. 
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Table  11.  Definitions  of  Reliability  Measures 


Measure 

Definition 

Percentage  of  Items 
Actually  Renewed  or 
Replaced 

The  percentage  of  items  actually  renewed  or  replaced  by  the  contractor.  A 
renewal/replacement  list  is  normally  created  by  the  contractor  with  the  consent  of 
the  Air  Force  a  year  in  advance  to  help  schedule  continuing  maintenance,  repairs, 
and  upgrades  for  the  installation  utility  system.  The  list  is  normally  executed  a  year 
later.  The  percentage  is  determined  by  the  ratio  of  the  actual  number  of  completed 
renewals  and  replacements  for  a  one-year  period  to  the  number  of  scheduled 
renewals  and  replacements  for  that  same  one-year  period. 

Percentage  of  Critical 
Outages  Caused  by 
System  Management 

The  percentage  of  critical  outages  caused  by  poor  system  management.  The 
percentage  is  determined  by  the  ratio  of  the  actual  number  of  hours  critical  facilities 
were  without  service  to  the  total  number  of  hours  critical  facilities  should  be  with 
service. 

Percentage  of  Non- 
Critical  Outages  Caused 
by  System  Management 

The  percentage  of  non-critical  outages  caused  by  poor  system  management.  The 
percentage  is  determined  by  the  ratio  of  the  actual  number  of  hours  non-critical 
facilities  were  without  service  to  the  total  number  of  hours  non-critical  facilities 
should  be  with  service. 

Table  12.  Definitions  of  Responsiveness  Measures 


Measure 

Definition 

Percentage  of  Items 
Actually  Renewed  or 
Replaced 

The  percentage  of  items  actually  renewed  or  replaced  by  the  contractor.  A 
renewal/replacement  list  is  normally  created  by  the  contractor  with  the  consent  of 
the  Air  Force  a  year  in  advance  to  help  schedule  continuing  maintenance,  repairs, 
and  upgrades  for  the  installation  utility  system.  The  list  is  normally  executed  a  year 
later.  The  percentage  is  determined  by  the  ratio  of  the  actual  number  of  completed 
renewals  and  replacements  for  a  one-year  period  to  the  number  of  scheduled 
renewals  and  replacements  for  that  same  one-year  period. 

Percentage  of  Critical 
Outages  Caused  by 
System  Management 

The  percentage  of  critical  outages  caused  by  poor  system  management.  The 
percentage  is  determined  by  the  ratio  of  the  actual  number  of  hours  critical  facilities 
were  without  service  to  the  total  number  of  hours  critical  facilities  should  be  with 
service. 

Percentage  of  Non- 
Critical  Outages  Caused 
by  System  Management 

The  percentage  of  non-critical  outages  caused  by  poor  system  management.  The 
percentage  is  determined  by  the  ratio  of  the  actual  number  of  hours  non-critical 
facilities  were  without  service  to  the  total  number  of  hours  non-critical  facilities 
should  be  with  service. 
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Table  12.  Definitions  of  Responsiveness  Measures  (Continued) 


Measure 

Definition 

Number  of 

Utility  Line  Hits 

The  number  of  utility  line  hits.  The  number  of  line  hits  is  an  indicator  of  the 
effectiveness  of  the  contractor's  digging  permit  and  line  marking  program. 

Percentage  of 
Meetings  Attended 

The  percentage  of  meetings  attended  by  the  contractor.  The  percentage  is 
determined  by  the  ratio  of  the  actual  number  of  meetings  attended  by  the  contractor 
by  the  number  of  meetings  scheduled  to  the  Air  Force. 

Percentage  of  Late 
Meter  Readings 

The  percentage  of  times  the  Air  Force  received  late  monthly  meter  readings  from  the 
contractor.  The  percentage  is  determined  by  the  ratio  of  the  actual  number  of 
months  the  meter  readings  were  late  to  12  months. 

Is  there  an  Adequate 
24/7  Hotline? 

Whether  the  contractor  has  adequate  24  hour  7  days  a  week  service  request  line 
support  for  base  personnel  to  call.  If  the  contractor  does  not  have  an  established 
telephone  available  for  all  base  personnel  to  call  24  hours  a  day,  7  days  a  week,  then 
the  contractor  is  considered  to  not  have  an  adequate  24/7  hotline. 

Percentage  of  Goal  Met 
for  Timely  Initial 
Emergency  Response 

The  percentage  of  goal  met  for  timely  initial  emergency  response.  A  representative 
for  the  contractor  is  required  to  be  on  the  site  of  the  emergency  response  within  one 
hour.  The  time  the  emergency  request  is  received  by  the  contractor  to  the  time  the 
contractor  sends  a  representative  is  normally  documented.  The  percentage  is 
determined  by  the  ratio  of  the  actual  number  of  times  the  representative  was  on  site 
within  one-hour  time  period  to  the  number  of  documented  emergency  requests. 

Percentage  of  Goal  for 
Timely  Emergency 
Crew  Response 

The  percentage  of  goal  met  for  timely  emergency  crew  response.  A  repair  crew 
consisting  of  qualified  technicians  working  for  the  contractor  is  required  to  be  on  the 
site  of  the  emergency  response  within  two  hours.  The  time  the  emergency  request  is 
received  by  the  contractor  and  the  time  the  contractor  sends  a  repair  crew  are 
normally  documented.  The  percentage  is  determined  by  the  ratio  of  the  actual 
number  of  times  the  repair  crew  was  on  site  within  a  two-hour  time  period  to  the 
number  of  documented  emergency  requests. 

Percentage  of  Goal  Met 
for  Timely  Remedied 
Emergency  Response 

The  percentage  of  goal  met  for  remedied  emergency  response.  A  repair  crew 
consisting  of  qualified  technicians  working  for  the  contractor  is  required  to  remedy 
or  downgrade  the  emergency  response  within  24  hours.  The  time  the  emergency 
request  is  received  by  the  contractor  to  the  time  the  repair  crew  remedies  or 
downgrades  the  emergency  response  is  normally  documented.  The  percentage  is 
determined  by  the  ratio  of  the  actual  number  of  times  the  repair  crew  remedies  or 
downgrades  the  emergency  response  within  a  24  hour  time  period  to  the  number  of 
documented  emergency  requests. 

Number  of  Outstanding 
Ratings  for 

Exercises/Contingencies 

The  degree  of  the  effectiveness  for  the  contractor’s  response  to  exercises  and 
contingencies.  The  rating  serves  as  an  indicator  of  the  effectiveness  of  the 
contractor’s  response  to  exercises  and  contingencies.  Outstanding  ratings  are 
observed  contractor  performance  or  operations  that  far  exceed  mission  requirements. 
The  contractor’s  procedures  and  activities  are  executed  in  a  far  superior  manner  with 
very  few  deficiencies,  if  any. 

Number  of  Excellent 
Ratings  for 

Exercises/Contingencies 

The  degree  of  the  effectiveness  for  the  contractor’s  response  to  exercises  and 
contingencies.  The  rating  serves  as  an  indicator  of  the  effectiveness  of  the 
contractor’s  response  to  exercises  and  contingencies.  Excellent  ratings  are  observed 
contractor  performance  or  operations  that  exceed  mission  requirements.  The 
contractor’s  procedures  and  activities  are  executed  in  a  superior  manner,  with  very 
little  deficiencies. 
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Table  12.  Definitions  of  Responsiveness  Measures  (Continued) 


Measure 

Definition 

Number  of  Satisfactory 
Ratings  for 

Exercises/Contingencies 

The  degree  of  the  effectiveness  for  the  contractor’s  response  to  exercises  and 
contingencies.  The  rating  serves  as  an  indicator  of  the  effectiveness  of  the 
contractor’s  response  to  exercises  and  contingencies.  Satisfactory  ratings  are 
observed  contractor  performance  or  operations  that  meet  mission  requirements.  The 
contractor’s  procedures  and  activities  are  executed  in  a  competent  manner,  with 
minor  deficiencies  that  so  not  impede  or  limit  the  mission. 

Number  of  Marginal 
Ratings  for 

Exercises/Contingencies 

The  degree  of  the  effectiveness  for  the  contractor’s  response  to  exercises  and 
contingencies.  The  rating  serves  as  an  indicator  of  the  effectiveness  of  the 
contractor’s  response  to  exercises  and  contingencies.  Marginal  ratings  are  observed 
contractor  performance  or  operations  that  meet  mission  requirements.  The 
contractor’s  procedures  and  activities  are  executed  in  an  inefficient  manner  with 
deficiencies  that  impede  or  limit  the  mission. 

Number  of 
Unsatisfactory 
Ratings  for 

Exercises/Contingencies 

The  degree  of  the  effectiveness  for  the  contractor’s  response  to  exercises  and 
contingencies.  The  rating  serves  as  an  indicator  of  the  effectiveness  of  the 
contractor’s  response  to  exercises  and  contingencies.  Unsatisfactory  ratings  are 
observed  contractor  performance  or  operations  that  do  not  meet  mission 
requirements.  The  contractor’s  procedures  and  activities  are  executed  in  an 
inadequate  manner  that  has  significant  deficiencies  that  seriously  impede  or  limit  the 
mission. 

Percentage  of  Goal  Met 
for  Effective  FACMAN 
Emergency 
Coordination 

The  percentage  of  goal  met  for  effective  FACMAN  emergency  coordination.  The 
contractor  is  required  to  coordinate  with  facility  managers  if  work  associated  with 
emergency  requests  will  affect  the  buildings.  The  names  of  facility  managers  or 
facility  manager  representatives  whom  were  contacted  are  normally  documented. 

The  percentage  is  determined  by  the  ratio  of  the  actual  number  of  times  the 
contractor  contacted  all  facility  managers  or  facility  manager  representatives 
associated  with  an  emergency  request  to  the  number  of  documented  emergency 
requests. 

Percentage  of  Goal  Met 
for  Timely  Initial 
Urgent  Response 

The  percentage  of  goal  met  for  timely  initial  urgent  response.  A  representative  for 
the  contractor  is  required  to  be  on  the  site  of  the  urgent  response  within  24  hours. 

The  time  the  urgent  request  is  received  by  the  contractor  to  the  time  the  contractor 
sends  a  representative  is  normally  documented.  The  percentage  is  determined  by  the 
ratio  of  the  actual  number  of  times  the  representative  was  on  site  within  a  24  hour 
time  period  to  the  number  of  documented  urgent  requests. 

Percentage  of  Goal  Met 
for  Timely  Remedied 
Urgent  Response 

The  percentage  of  goal  met  for  remedied  urgent  response.  A  repair  crew  consisting 
of  qualified  technicians  working  for  the  contractor  is  required  to  remedy  or 
downgrade  the  emergency  response  within  five  calendar  days.  The  time  the  urgent 
request  is  received  by  the  contractor  to  the  time  the  repair  crew  remedies  or 
downgrades  the  urgent  response  is  normally  documented.  The  percentage  is 
determined  by  the  ratio  of  the  actual  number  of  times  the  repair  crew  remedies  or 
downgrades  the  urgent  response  within  a  five  calendar  day  time  period  to  the 
number  of  documented  urgent  requests. 
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Table  12.  Definitions  of  Responsiveness  Measures  (Continued) 


Measure 

Definition 

Percentage  of  Goal  Met 
for  Effective  FACMAN 
Urgent  Coordination 

The  percentage  of  goal  met  for  effective  FACMAN  emergency  coordination.  The 
contractor  is  required  to  coordinate  with  facility  managers  if  work  associated  with 
emergency  requests  will  affect  the  buildings.  The  names  of  facility  managers  or 
facility  manager  representatives  whom  were  contacted  are  normally  documented. 

The  percentage  is  determined  by  the  ratio  of  the  actual  number  of  times  the 
contractor  contacted  all  facility  managers  or  facility  manager  representatives 
associated  with  an  emergency  request  to  the  number  of  documented  urgent  requests. 

Percentage  of  Goal  Met 
for  Timely  Initial 
Routine  Response 

The  percentage  of  goal  met  for  timely  initial  routine  response.  A  representative  for 
the  contractor  is  required  to  be  on  the  site  of  the  emergency  response  within  five 
calendar  days.  The  time  the  routine  request  is  received  by  the  contractor  to  the  time 
the  contractor  sends  a  representative  is  normally  documented.  The  percentage  is 
determined  by  the  ratio  of  the  actual  number  of  times  the  representative  was  on  site 
within  a  one-hour  time  period  to  the  number  of  documented  routine  requests. 

Percentage  of  Goal  Met 
for  Timely  Remedied 
Routine  Response 

The  percentage  of  goal  met  for  remedied  routine  response.  A  repair  crew  consisting 
of  qualified  technicians  working  for  the  contractor  is  required  to  remedy  or 
downgrade  the  emergency  response  within  30  calendar  days.  Both  the  time  the 
routine  request  is  received  by  the  contractor  and  the  time  the  repair  crew  remedies 
the  routine  response  are  normally  documented.  The  percentage  is  determined  by  the 
ratio  of  the  actual  number  of  times  the  repair  crew  remedies  the  routine  response 
within  a  five  calendar  day  time  period  to  the  number  of  documented  routine 
requests. 

Percentage  of  Goal  Met 
for  Effective  2-Week 
Coordination 

The  percentage  of  goal  met  for  effective  two-week  coordination  with  contracting 
office  representative.  The  contractor  is  required  to  coordinate  with  the  installation's 
contracting  office  representative  at  least  two  weeks  prior  to  commencing  work  for  a 
routine  service  request.  The  names  of  contracting  office  representatives  contacted 
are  normally  documented.  The  percentage  is  determined  by  the  ratio  of  the  actual 
number  of  times  the  contractor  contacted  a  contracting  office  representative 
associated  with  a  routine  service  request  to  the  number  of  routine  service  requests. 

Percentage  of  Goal  Met 
for  Effective  FACMAN 
Routine  Coordination 

The  percentage  of  goal  met  for  effective  FACMAN  routine  coordination.  The 
contractor  is  required  to  coordinate  with  facility  managers  if  work  associated  with 
routine  requests  will  affect  the  buildings.  The  names  of  facility  managers  or  their 
representatives  contacted  are  normally  documented.  The  percentage  is  determined 
by  the  ratio  of  the  actual  number  of  times  the  contactor  contacted  all  facility 
managers  or  facility  manager  representatives  associated  with  a  routine  request  to  the 
number  of  routine  requests. 
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Appendix  C.  Single  Dimension  Value  Functions  (SDVFs) 


SDVF  #  1 — Percentage  of  Up-to-Date  Licenses,  Permits,  and  Certifications 

The  SDVF  for  Percentage  of  Up-to-Date  Licenses,  Permits,  and  Certifications  is  a 
monotonically  increasing  function  that  measures  the  contractor’s  percentage  of  up-to-date 
licenses,  permits,  and  certifications.  The  most  preferred  score  is  for  a  contractor  that  was 
able  to  keep  all  licenses,  permits,  and  certifications  up-to-date.  This  score  receives  a 
value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  that  unable  to  keep  none 
licenses,  permits,  and  certification  up-to-date.  That  score  receives  a  value  of  0.000.  As 
illustrated  in  Figure  30,  the  contractor’s  value  on  the  y-axis  increases  for  every  increase 
in  percentage  on  the  x-axis. 


Figure  30.  Percentage  of  Up-to-Date  Licenses,  Permits,  and  Certifications  SDVF 
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SDVF  #  2 — Are  Records  for  the  Past  2  Years  Maintained  Properly 

The  SDVF  for  Are  Records  for  the  Past  2  Years  Maintained  Properly  is  a  discrete 
function  that  measures  if  a  contractor  is  properly  maintaining  service  records  for  the  past 
two  years.  The  most  preferred  score  is  for  a  contractor  that  was  able  to  properly  maintain 
all  service  records  for  the  past  two  years.  This  score  receives  a  value  of  1 .000.  The  least 
preferred  score  is  for  a  contractor  that  unable  to  properly  maintain  all  service  records  for 
the  past  two  years.  That  score  receives  a  value  of  0.000.  The  Are  Records  for  the  Past  2 
Years  Maintained  Properly  SDVF  is  illustrated  in  Figure  3 1 . 


Label  Value 
Yes 

No  0.000 

Figure  3 1 .  Are  Records  for  the  Past  2  Years  Maintained  Properly  SDVF 
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SDVF  #  3 — Average  Number  of  Days  to  Update  Drawings 


The  SDVF  for  Average  Number  of  Days  to  Update  Drawings  is  a  discrete  function 
that  measures  how  long  it  takes  a  contractor  to  update  drawings.  The  most  preferred 
score  is  for  a  contractor  that  was  able  to  update  drawings  under  60  days.  This  score 
receives  a  value  of  1 .000.  The  second  most  preferred  score  is  for  a  contractor  that  was 
able  to  update  drawings  between  60  to  65  days.  That  score  receives  a  value  of  0.838. 

The  third  most  preferred  score  is  for  a  contractor  for  a  contractor  that  was  able  to  update 
drawings  between  66  to  70  days.  This  score  receives  a  value  of  0.420.  The  least 
preferred  score  is  for  a  contractor  that  was  only  able  to  update  drawings  over  75  days. 
That  score  receives  a  value  of  0.000.  The  second  least  preferred  score  is  for  a  contractor 
that  was  only  able  to  update  drawings  between  71  to  75  days.  This  score  receives  a  value 
of  0. 1 80.  The  Average  Number  of  Days  to  Update  Drawings  SDVF  is  illustrated  in 
Figure  32. 


Label  Value 


Under  60  days 
60  -  65  days 
66-  70  days 
71  -  75  days 
Over  75  days 


1.000 

0.838 

0.420 

0.180 

0.000 


Figure  32.  Average  Number  of  Days  to  Update  Drawings  SDVF 
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SDVF  #  4 — Number  of  Positive  Findings  for  Spill  Contingency  Plan 

The  SDVF  for  Number  of  Positive  Findings  for  Spill  Contingency  Plan  is  a 
monotonically  increasing  function  that  measures  the  number  of  positive  findings  the 
contractor  received  for  its  spill  contingency  plan.  The  most  preferred  score  is  for  a 
contractor  that  received  over  up  to  20  positive  findings.  This  score  receives  a  value  of 
1 .000.  The  least  preferred  score  is  for  a  contractor  that  received  no  positive  findings. 
That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  33,  the  contractor’s  value  on 
the  y-axis  increases  for  every  increase  in  the  number  of  findings  on  the  x-axis. 
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SDVF  #  5 — Number  of  Minor  Findings  for  Spill  Contingency  Plan 

The  SDVF  for  Number  of  Minor  Findings  for  Spill  Contingency  Plan  is  a 
monotonically  decreasing  function  that  measures  the  number  of  minor  findings  the 
contractor  received  for  its  spill  contingency  plan.  The  most  preferred  score  is  for  a 
contractor  that  received  no  minor  findings.  This  score  receives  a  value  of  1.000.  The 
least  preferred  score  is  for  a  contractor  that  received  up  to  20  minor  findings.  That  score 
receives  a  value  of  0.000.  As  illustrate  in  Figure  34,  the  contractor’s  value  on  the  y-axis 
decreases  for  every  increase  in  the  number  of  findings  on  the  x-axis. 
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SDVF  #  6 — Number  of  Major  Findings  for  Spill  Contingency  Plan 

The  SDVF  for  Number  of  Major  Findings  for  Spill  Contingency  Plan  is  a 
monotonically  decreasing  function  that  measures  the  number  of  major  findings  the 
contractor  received  for  its  spill  contingency  plan.  The  most  preferred  score  is  for  a 
contractor  that  received  no  major  findings.  This  score  receives  a  value  of  1 .000.  The 
least  preferred  score  is  for  a  contractor  that  received  up  to  20  major  findings.  That  score 
receives  a  value  of  0.000.  As  illustrate  in  Figure  35,  the  contractor’s  value  on  the  y-axis 
decreases  for  every  increase  in  the  number  of  findings  on  the  x-axis. 
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SDVF  #  7 — Number  of  Significant  Findings  for  Spill  Contingency  Plan 

The  SDVF  for  Number  of  Significant  Findings  for  Spill  Contingency  Plan  is  a 
monotonically  decreasing  function  that  measures  the  number  of  significant  findings  the 
contractor  received  for  its  spill  contingency  plan.  The  most  preferred  score  is  for  a 
contractor  that  received  no  significant  findings.  This  score  receives  a  value  of  1.000. 

The  least  preferred  score  is  for  a  contractor  that  received  up  to  20  significant  findings. 
That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  36,  the  contractor’s  value  on 
the  y-axis  decreases  for  every  increase  in  the  number  of  findings  on  the  x-axis. 
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SDVF  #  8 — Percentage  of  Liquid  Waste  Diverted  from  Landfills 

The  SDVF  for  Percentage  of  Liquid  Waste  Diverted  from  Landfills  is  a 
monotonically  increasing  function  that  measures  the  contractor’s  percentage  of  its  liquid 
waste  diverted  from  landfills.  The  most  preferred  score  is  for  a  contractor  that  was  able 
to  divert  all  of  its  liquid  waste  from  landfills.  This  score  receives  a  value  of  1.000.  The 
least  preferred  score  is  for  a  contractor  that  was  not  able  to  divert  any  of  its  liquid  waste 
from  landfills.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  37,  the 
contractor’s  value  on  the  y-axis  increases  for  every  increase  in  percentage  on  the  x-axis. 
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SDVF  #  9 — Percentage  of  Solid  Waste  Diverted  from  Landfills 


The  SDVF  for  Percentage  of  Solid  Waste  Diverted  from  Landfills  is  a 
monotonically  increasing  function  that  measures  the  contractor’s  percentage  of  its  solid 
waste  diverted  from  landfills.  The  most  preferred  score  is  for  a  contractor  that  was  able 
to  divert  all  of  its  liquid  waste  from  landfills.  This  score  receives  a  value  of  1.000.  The 
least  preferred  score  is  for  a  contractor  that  was  unable  to  divert  any  of  its  liquid  waste 
from  landfills.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  38,  the 
contractor’s  value  on  the  y-axis  increases  for  every  increase  in  percentage  on  the  x-axis. 
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SDVF  #  10 — Number  of  Positive  Findings  for  Recycling 


The  SDVF  for  Number  of  Positive  Findings  for  Recycling  is  a  monotonically 
increasing  function  that  measures  the  number  of  positive  findings  the  contractor  received 
for  its  contribution  to  the  installation’s  recycling  program.  The  most  preferred  score  is 
for  a  contractor  that  received  up  to  20  positive  findings.  This  score  receives  a  value  of 
1 .000.  The  least  preferred  score  is  for  a  contractor  that  received  no  positive  findings. 
That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  39,  the  contractor’s  value  on 
the  y-axis  increases  for  every  increase  in  the  number  of  findings  on  the  x-axis. 
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SDVF  #  11 — Number  of  Minor  Findings  for  Recycling 


The  SDVF  for  Number  of  Minor  Findings  for  Recycling  is  a  monotonically 
decreasing  function  that  measures  the  number  of  minor  findings  the  contractor  received 
for  its  contribution  to  the  installation’s  recycling  program.  The  most  preferred  score  is 
for  a  contractor  that  received  no  minor  findings.  This  score  receives  a  value  of  1.000. 
The  least  preferred  score  is  for  a  contractor  that  received  up  to  20  minor  findings.  That 
score  receives  a  value  of  0.000.  As  illustrate  in  Figure  40,  the  contractor’s  value  on  the 
y-axis  decreases  for  every  increase  in  the  number  of  findings  on  the  x-axis. 
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SDVF  #  12 — Number  of  Major  Findings  for  Recycling 


The  SDVF  for  Number  of  Major  Findings  for  Recycling  is  a  monotonically 
decreasing  function  that  measures  the  number  of  major  findings  the  contractor  received 
for  its  contribution  to  the  installation’s  recycling  program.  The  most  preferred  score  is 
for  a  contractor  that  received  no  major  findings.  This  score  receives  a  value  of  1.000. 
The  least  preferred  score  is  for  a  contractor  that  received  up  to  20  major  findings.  That 
score  receives  a  value  of  0.000.  As  illustrate  in  Figure  41,  the  contractor’s  value  on  the 
y-axis  exponentially  decreases  for  every  increase  in  the  number  of  findings  on  the  x-axis 
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SDVF  #  13 — Number  of  Significant  Findings  for  Recycling 


The  SDVF  for  Number  of  Significant  Findings  for  Recycling  is  a  monotonically 
decreasing  function  that  measures  the  number  of  significant  findings  the  contractor 
received  for  its  contribution  to  the  installation’s  recycling  program.  The  most  preferred 
score  is  for  a  contractor  that  received  no  significant  findings.  This  score  receives  a  value 
of  1 .000.  The  least  preferred  score  is  for  a  contractor  that  received  up  to  20  significant 
findings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  42,  the  contractor’s 
value  on  the  y-axis  decreases  for  every  increase  in  the  number  of  findings  on  the  x-axis. 
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SDVF  #  14 — Number  of  Lost  Man-Hours  Due  to  Utility  System  Mishaps 

The  SDVF  for  Number  of  Lost  Man-Hours  Due  to  Utility  System  Mishaps  is  a 
monotonically  decreasing  function  that  measures  the  number  of  lost  man-hours  due  to  the 
contractor’s  utility  system  mishaps.  The  most  preferred  score  is  for  a  contractor  with  a 
utility  system  that  caused  no  lost  in  man-hours.  This  score  receives  a  value  of  1.000. 

The  least  preferred  score  is  for  a  contractor  with  a  utility  system  that  caused  up  to  150 
lost  man-hours.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  43,  the 
contractor’s  value  on  the  y-axis  decreases  for  every  increase  in  the  number  of  man-hours 
on  the  x-axis. 
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SDVF  #  15 — Number  of  Utility  System  Mishaps 


The  SDVF  for  Number  of  Utility  System  Mishaps  is  a  monotonically  decreasing 
function  that  measures  the  number  of  utility  system  mishaps  caused  by  the  contractor’s 
safety  practices.  The  most  preferred  score  is  for  a  contractor  whose  safety  practices 
caused  no  utility  system  mishaps.  This  score  receives  a  value  of  1.000.  The  least 
preferred  score  is  for  a  contractor  whose  safety  practices  caused  up  to  15  utility  system 
mishaps.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  44,  the  contractor’s 
value  on  the  y-axis  decreases  for  every  increase  in  the  number  of  mishaps  on  the  x-axis. 
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SDVF  #  16 — Number  of  RAC  1  -  Catastrophic  Violations 


The  SDVF  for  Number  of  RAC  1  -  Catastrophic  Violations  is  a  monotonically 
decreasing  function  which  measures  the  number  of  catastrophic  violations  caused  by  the 
contractor’s  safety  practices.  The  most  preferred  score  is  for  a  contractor  whose  safety 
practices  caused  no  catastrophic  violations.  This  score  receives  a  value  of  1.000.  The 
least  preferred  score  is  for  a  contractor  whose  safety  practices  caused  up  to  10 
catastrophic  violations.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  45, 
the  contractor’s  value  on  the  y-axis  decreases  for  every  increase  in  the  number  of 
violations  on  the  x-axis. 
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SDVF  #  17 — Number  of  RAC  2  -  Critical  Violations 


The  SDVF  for  Number  of  RAC  2  -  Critical  Violations  is  a  monotonically 
decreasing  function  which  measures  the  number  of  critical  violations  caused  by  the 
contractor’s  safety  practices.  The  most  preferred  score  is  for  a  contractor  whose  safety 
practices  caused  no  critical  violations.  This  score  receives  a  value  of  1.000.  The  least 
preferred  score  is  for  a  contractor  whose  safety  practices  caused  up  to  10  critical 
violations.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  46,  the 
contractor’s  value  on  the  y-axis  decreases  for  every  increase  in  the  number  of  violations 
on  the  x-axis. 
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SDVF  #  18 — Number  of  RAC  3  -  Moderate  Violations 


The  SDVF  for  Number  RAC  3  -  Moderate  Violations  is  a  monotonically  decreasing 
function  which  measures  the  number  of  moderate  violations  caused  by  the  contractor’s 
safety  practices.  The  most  preferred  score  is  for  a  contractor  whose  safety  practices 
caused  no  moderate  violations.  This  score  receives  a  value  of  1.000.  The  least  preferred 
score  is  for  a  contractor  whose  safety  practices  caused  up  to  10  moderate  violations.  That 
score  receives  a  value  of  0.000.  As  illustrate  in  Figure  47,  the  contractor’s  value  on  the 
y-axis  decreases  for  every  increase  in  the  number  of  violations  on  the  x-axis. 
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SDVF  #  19 — Number  of  RAC  4  -  Negligible  Violations 


The  SDVF  for  Number  of  RAC  4  -  Negligible  Violations  is  a  monotonically 
decreasing  function  which  measures  the  number  of  negligible  violations  caused  by  the 
contractor’s  safety  practices.  The  most  preferred  score  is  for  a  contractor  whose  safety 
practices  caused  no  negligible  violations.  This  score  receives  a  value  of  1.000.  The  least 
preferred  score  is  for  a  contractor  whose  safety  practices  caused  up  to  10  negligible 
violations.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  48,  the 
contractor’s  value  on  the  y-axis  decreases  for  every  increase  in  the  number  of  violations 
on  the  x-axis. 
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SDVF  #  20 — Percentage  of  Employees  Completing  all  Requirements 

The  SDVF  for  Percentage  of  Employees  Completing  all  Requirements  is  a 
monotonically  increasing  function  that  measures  the  percentage  employees  working  for 
the  contractor  whom  completed  all  safety  certification  requirements.  The  most  preferred 
score  is  for  a  contractor  whose  required  employees  are  fully  certified.  This  score  receives 
a  value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  whose  required  employees 
are  not  fully  certified.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  49, 
the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in  percentage  on  the  x- 
axis. 
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SDVF  #  21 — Percentage  of  Meters  Calibrated  from  Random  Sample 

The  SDVF  for  Percentage  of  Meters  Calibrated  from  Random  Sample  is  a 
monotonically  increasing  function  which  measures  the  percentage  of  a  random  sample  of 
meters  maintained  by  the  contractor  that  are  calibrated.  The  most  preferred  score  is  for 
all  meters  in  the  random  sample  to  be  calibrated.  This  score  receives  a  value  of  1.000. 
The  least  preferred  score  is  for  none  of  the  meters  in  the  random  sample  to  be  calibrated. 
That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  50,  the  contractor’s  value  on 
the  y-axis  increases  for  every  increase  in  percentage  on  the  x-axis. 
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SDVF  #  22 — Percentage  of  Total  Facilities  Metered 


The  SDVF  for  Percentage  of  Total  Facilities  Metered  is  a  monotonically  increasing 
function  that  measures  the  percentage  of  facilities  on  the  installation  that  are  metered. 

The  most  preferred  score  is  for  all  the  facilities  on  the  installations  to  be  metered.  This 
score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  none  of  the  facilities  on 
the  installation  to  be  metered.  That  score  receives  a  value  of  0.000.  As  illustrate  in 
Figure  51,  the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in  percentage 
on  the  x-axis. 
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SDVF  #  23 — Number  of  Employees  Identified  as  Potential  Threats 

The  SDVF  for  Number  of  Employees  Identified  as  Potential  Threats  is  a  discrete 
function  that  measures  number  of  contractor  employees  that  are  potential  security  threats 
to  the  installation.  The  most  preferred  score  is  for  a  contractor  with  no  employees 
identified  as  a  potential  security  threat.  This  score  receives  a  value  of  1.000.  The  least 
preferred  score  is  for  a  contractor  with  more  than  one  employee  identified  as  a  potential 
security  threat.  That  score  receives  a  value  of  0.000.  The  second  least  preferred  score  is 
for  a  contractor  that  has  only  one  employee  identified  as  a  potential  security  threat.  This 
score  receives  a  value  of  0.400.  The  Number  of  Employees  Identified  as  Potential 
Threats  SDVF  is  illustrated  in  Figure  52. 


Label 

Value 

None  Identified 

1  Identified 

More  than  1 

0.000 

Figure  52.  Number  of  Employees  Identified  as  Potential  Threats  SDVF 
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SDVF  #  24 — Are  all  Employee  Security  Clearances  Up-to-Date 


The  SDVF  for  Are  all  Employee  Security  Clearances  Up-to-Date  is  a  discrete 
function,  which  measures  whether  all  required  contractor  employees  have  up-to-date 
security  clearances.  The  most  preferred  score  is  for  a  contractor  with  employees 
requiring  security  clearances  have  them  up-to-date.  This  score  receives  a  value  of  1.000. 
The  least  preferred  score  is  for  a  contractor  with  employees  requiring  security  clearances 
not  have  them  fully  up-to-date.  That  score  receives  a  value  of  0.000.  The  Are  all 
Employee  Security  Clearances  Up-to-Date  SDVF  is  illustrated  in  Figure  53. 


Label  Value 
Yes  1-000 

No  0.000 


Figure  53.  Are  all  Employee  Security  Clearances  Up-to-Date  SDVF 
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SDVF  #  25 — Percentage  of  Items  Actually  Replaced 


The  SDVF  for  Percentage  of  Items  Actually  Replaced  is  a  monotonically  increasing 
function  that  measures  the  percentage  of  items  on  the  renewals/replacement  list  actually 
completed  by  the  contractor.  The  most  preferred  score  is  for  the  contractor  that 
completed  all  items  on  the  list.  This  score  receives  a  value  of  1.000.  The  least  preferred 
score  is  for  contractor  that  completed  no  items  on  the  list.  That  score  receives  a  value  of 
0.000.  As  illustrate  in  Figure  54,  the  contractor’s  value  on  the  y-axis  increases  for  every 
increase  in  percentage  on  the  x-axis. 
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SDVF  #  26 — Percentage  of  Critical  Outages  Caused  by  System  Management 

The  SDVF  for  Percentage  of  Critical  Outages  Caused  by  System  Management  is  a 
monotonically  decreasing  function  that  measures  the  percentage  of  critical  facility 
outages  caused  by  the  contractor’s  system  management.  The  most  preferred  score  is  for  a 
contractor  with  no  critical  facility  outages  caused  by  its  system  management.  This  score 
receives  a  value  of  1.000.  The  least  preferred  score  is  for  a  contractor  with  all  critical 
facility  outages  caused  by  its  system  management.  That  score  receives  a  value  of  0.000. 
As  illustrate  in  Figure  55,  the  contractor’s  value  on  the  y-axis  decreases  for  every 
increase  in  percentage  on  the  x-axis. 
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SDVF  #  27 — Percentage  of  Non-Critical  Outages  Caused  by  System  Management 


The  SDVF  for  Percentage  of  Non-Critical  Outages  Caused  by  System  Management 
is  a  mono  tonic  ally  decreasing  function  that  measures  the  percentage  of  non-critical 
facility  outages  caused  by  the  contractor’s  system  management.  The  most  preferred 
score  is  for  a  contractor  with  no  non-critical  facility  outages  caused  by  its  system 
management.  This  score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  a 
contractor  with  all  non-critical  facility  outages  caused  by  its  system  management.  That 
score  receives  a  value  of  0.000.  As  illustrate  in  Figure  56,  the  contractor’s  value  on  the 
y-axis  decreases  for  every  increase  in  percentage  on  the  x-axis. 
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SDVF  #  28 — Number  of  Utility  Line  Hits 


The  SDVF  for  Number  of  Utility  Line  Hits  is  a  discrete  function  which  measures 
how  number  of  utility  line  hits  caused  by  the  contractor’s  digging  permit  and  line 
marking  program.  The  most  preferred  score  is  for  a  contractor  to  have  no  utility  line 
hits.  This  score  receives  a  value  of  1.000.  The  second  most  preferred  score  is  for  a 
contractor  to  have  between  1  to  5  utility  line  hits.  That  score  receives  a  value  of  0.670. 
The  least  preferred  score  is  for  a  contractor  to  have  more  than  10  utility  line  hits.  That 
score  receives  a  value  of  0.000.  The  second  least  preferred  score  is  for  a  contractor  to 
have  between  6  to  10  utility  line  hits.  This  score  receives  a  value  of  0.333.  The  Number 
of  Utility  Line  Hits  SDVF  is  illustrated  in  Figure  57. 


Label 

Value 

OHits 

1  to  5  Flits 

6  to  1 0  Flits 

Greater  than  10  Flits 

0.000 

Figure  57.  Number  of  Utility  Line  Flits  SDVF 
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SDVF  #  29 — Percentage  of  Meetings  Attended 


The  SDVF  for  Percentage  of  Items  Actually  Replaced  is  a  monotonically  increasing 
function  which  measures  the  percentage  of  Air  Force  requested  meetings  attended  by  the 
contractor.  The  most  preferred  score  is  for  the  contractor  that  attended  all  meetings.  This 
score  receives  a  value  of  1 .000.  The  least  preferred  score  is  for  contractor  that  was 
unable  to  attend  meetings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure 
58,  the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in  percentage  on  the 
x-axis. 
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SDVF  #  30 — Percentage  of  Late  Meter  Readings 


The  SDVF  for  Percentage  of  Late  Meter  Readings  is  a  monotonically  decreasing 
function  that  measures  the  percentage  of  times  the  Air  Force  received  late  meter  readings 
from  the  contractor.  The  most  preferred  score  is  for  a  contractor  to  provide  all  meter 
readings  on  time  to  the  Air  Force.  This  score  receives  a  value  of  1.000.  The  least 
preferred  score  is  for  a  contractor  to  provide  the  Air  Force  with  consistently  late  meter 
readings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  59,  the 
contractor’s  value  on  the  y-axis  exponentially  decreases  for  every  increase  in  percentage 
on  the  x-axis. 
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SDVF  #  31 — Is  There  an  Adequate  24/7  Hotline 


The  SDVF  for  Is  There  an  Adequate  24/7  Hotline  is  a  discrete  function  which 
measures  if  a  contractor  has  adequate  hotline  support  for  base  personnel  to  call  24  hours  a 
day,  7  days  a  week.  The  most  preferred  score  is  for  a  contractor  has  adequate  hotline 
support  for  base  personnel  to  call  24  hours  a  day,  7  days  a  week.  This  score  receives  a 
value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  that  is  unable  to  provide  base 
personnel  with  adequate  hotline  support  24  hours  a  day,  7  days  a  week.  That  score 
receives  a  value  of  0.000.  The  Is  There  an  Adequate  24/7  Hotline  SDVF  is  illustrated  in 
Figure  60. 


Label  Value 

Yes  i-000 
No  0.000 


Figure  60.  Is  There  an  Adequate  24/7  Hotline  SDVF 
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SDVF  #  32 — Percentage  of  Goal  Met  for  Timely  Initial  Emergency  Response 

The  SDVF  for  Percentage  of  Goal  Met  for  Timely  Initial  Emergency  Response  is  a 
monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  initial  emergency  response  goal.  The  most  preferred  score  is  for  a  contractor  that 
was  able  to  have  a  representative  on  site  to  respond  to  every  initial  emergency  response 
in  a  timely  manner.  This  score  receives  a  value  of  1.000.  The  least  preferred  score  is  for 
a  contractor  that  was  unable  to  have  a  representative  on  site  to  respond  to  initial 
emergency  responses  in  a  timely  manner.  That  score  receives  a  value  of  0.000.  As 
illustrate  in  Figure  61,  the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in 
percentage  on  the  x-axis. 
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SDVF  #  33 — Percentage  of  Goal  Met  for  Timely  Emergency  Crew  Response 

The  SDVF  for  Percentage  of  Goal  Met  for  Timely  Emergency  Crew  Response  is  a 
monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  emergency  crew  response  goal.  The  most  preferred  score  is  for  a  contractor 
whose  crews  were  able  to  respond  to  all  emergency  responses  in  a  timely  manner.  This 
score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  a  contractor  whose  crews 
were  unable  to  respond  to  emergency  responses  in  a  timely  manner.  That  score  receives 
a  value  of  0.000.  As  illustrate  in  Figure  62,  the  contractor’s  value  on  the  y-axis  increases 
for  every  increase  in  percentage  on  the  x-axis. 


143 


SDVF  #  34 — Percentage  of  Goal  Met  for  Timely  Remedied  Emergency  Response 


The  SDVF  for  Percentage  of  Goal  Met  for  Timely  Remedied  Emergency  Response 
is  a  monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  remedied  emergency  response  goal.  The  most  preferred  score  is  for  a  contractor 
whose  crews  were  able  to  remedy  all  emergency  responses  in  a  timely  manner.  This 
score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  a  contractor  whose  crews 
were  unable  to  remedy  emergency  responses  in  a  timely  manner.  That  score  receives  a 
value  of  0.000.  As  illustrate  in  Figure  63,  the  contractor’s  value  on  the  y-axis  increases 
for  every  increase  in  percentage  on  the  x-axis. 


Figure  63.  Percentage  of  Goal  Met  for  Timely  Remedied  Emergency  Response  SDVF 
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SDVF  #  35 — Number  of  Outstanding  Ratings  for  Exercises/Contingencies 

The  SDVF  for  Number  of  Outstanding  Ratings  for  Exercise/Contingencies  a 
monotonically  increasing  function  that  measures  the  number  of  outstanding  ratings  the 
contractor  received  for  its  responsiveness  to  installation  exercises  and  contingencies.  The 
most  preferred  score  is  for  a  contractor  that  received  up  to  20  outstanding  ratings.  This 
score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  a  contractor  that  received 
no  outstanding  ratings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  64, 
the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in  the  number  of  ratings 
on  the  x-axis. 
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SDVF  #  36 — Number  of  Excellent  Ratings  from  Exercises/Contingencies 

The  SDVF  for  Number  of  Excellent  Ratings  for  Exercise/Contingencies  a 
monotonically  increasing  function  that  measures  the  number  of  excellent  ratings  the 
contractor  received  for  its  responsiveness  to  installation  exercises  and  contingencies.  The 
most  preferred  score  is  for  a  contractor  that  received  up  to  20  excellent  ratings.  This 
score  receives  a  value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  that  received 
no  excellent  ratings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  65,  the 
contractor’s  value  on  the  y-axis  increases  for  every  increase  in  the  number  of  ratings  on 
the  x-axis. 


Figure  65.  Number  of  Excellent  Ratings  from  Exercises/Contingencies  SDVF 
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SDVF  #  37 — Number  of  Satisfactory  Ratings  from  Exercises/Contingencies 

The  SDVF  for  Number  of  Satisfactory  Ratings  for  Exercise/Contingencies  a 
monotonically  increasing  function  that  measures  the  number  of  satisfactory  ratings  the 
contractor  received  for  its  responsiveness  to  installation  exercises  and  contingencies.  The 
most  preferred  score  is  for  a  contractor  that  received  up  to  20  satisfactory  ratings.  This 
score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  a  contractor  that  received 
no  satisfactory  ratings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  66, 
the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in  the  number  of  ratings 
on  the  x-axis. 


Figure  66.  Number  of  Excellent  Ratings  from  Exercises/Contingencies  SDVF 
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SDVF  #  38 — Number  of  Marginal  Ratings  from  Exercises/Contingencies 

The  SDVF  for  Number  of  Marginal  Ratings  from  Exercises/Contingencies  is  a 
monotonically  decreasing  function  that  measures  the  number  of  marginal  ratings  the 
contractor  received  for  its  responsiveness  to  installation  exercises  and  contingencies.  The 
most  preferred  score  is  for  a  contractor  that  received  no  marginal  ratings.  This  score 
receives  a  value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  that  received  up  to 
20  marginal  ratings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  67,  the 
contractor’s  value  on  the  y-axis  decreases  for  every  increase  in  the  number  of  ratings  on 
the  x-axis. 


Figure  67.  Number  of  Marginal  Ratings  from  Exercises/Contingencies  SDVF 
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SDVF  #  39 — Number  of  Unsatisfactory  Ratings  from  Exercises/Contingencies 

The  SDVF  for  Number  of  Unsatisfactory  Ratings  from  Exercises/Contingencies  is  a 
monotonically  decreasing  function  that  measures  the  number  of  unsatisfactory  ratings  the 
contractor  received  for  its  responsiveness  to  installation  exercises  and  contingencies.  The 
most  preferred  score  is  for  a  contractor  that  received  no  unsatisfactory  ratings.  This  score 
receives  a  value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  that  received  up  to 
20  unsatisfactory  ratings.  That  score  receives  a  value  of  0.000.  As  illustrate  in  Figure  68, 
the  contractor’s  value  on  the  y-axis  decreases  for  every  increase  in  the  number  of  ratings 
on  the  x-axis. 


Figure  68.  Number  of  Unsatisfactory  Ratings  from  Exercises/Contingencies  SDVF 
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SDVF  #  40 — Percentage  of  Goal  Met  for  Effective  FACMAN  Emergency 
Coordination 

The  SDVF  for  Percentage  of  Goal  Met  for  Effective  FACMAN  Emergency 
Coordination  is  a  monotonically  increasing  function  that  measures  the  percentage  of  time 
the  contractor  met  the  FACMAN  emergency  coordination  goal.  The  most  preferred  score 
is  for  a  contractor  that  was  able  to  coordinate  every  emergency  response  with  the  person 
responsible  for  the  building  or  facility  it  affects.  This  score  receives  a  value  of  1.000. 

The  least  preferred  score  is  for  a  contractor  that  was  unable  to  coordinate  emergency 
responses  with  the  person  responsible  for  the  building  or  facility  it  affects.  That  score 
receives  a  value  of  0.000.  As  illustrate  in  Figure  69,  the  contractor’s  value  on  the  y-axis 
increases  for  every  increase  in  percentage  on  the  x-axis. 


Figure  69.  Percentage  of  Goal  Met  for  Effective  FACMAN  Emergency  Coordination 

SDVF 
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SDVF  #  41 — Percentage  of  Goal  Met  for  Timely  Initial  Urgent  Response 

The  SDVF  for  Percentage  of  Goal  Met  for  Timely  Initial  Urgent  Response  is  a 
monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  initial  urgent  response  goal.  The  most  preferred  score  is  for  a  contractor  that  was 
able  to  have  a  representative  on  site  to  respond  to  every  initial  urgent  response  in  a  timely 
manner.  This  score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  a 
contractor  that  was  unable  to  have  a  representative  on  site  to  respond  to  initial  urgent 
responses  in  a  timely  manner.  That  score  receives  a  value  of  0.000.  As  illustrate  in 
Figure  70,  the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in  percentage 
on  the  x-axis. 
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SDVF  #  42 — Percentage  of  Goal  Met  for  Timely  Remedied  Urgent  Response 

The  SDVF  for  Percentage  of  Goal  Met  for  Timely  Remedied  Urgent  Response  is  a 
monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  remedied  urgent  response  goal.  The  most  preferred  score  is  for  a  contractor 
whose  crews  were  able  to  remedy  all  urgent  responses  in  a  timely  manner.  This  score 
receives  a  value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  whose  crews  were 
unable  to  remedy  urgent  responses  in  a  timely  manner.  That  score  receives  a  value  of 
0.000.  As  illustrate  in  Figure  71,  the  contractor’s  value  on  the  y-axis  increases  for  every 
increase  in  percentage  on  the  x-axis. 
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SDVF  #  43 — Percentage  of  Goal  Met  for  Effective  FACMAN  Urgent  Coordination 


The  SDVF  for  Percentage  of  Goal  Met  for  Effective  FACMAN  Urgent 
Coordination  is  a  monotonically  increasing  function  that  measures  the  percentage  of  time 
the  contractor  met  the  FACMAN  urgent  coordination  goal.  The  most  preferred  score  is 
for  a  contractor  that  was  able  to  coordinate  every  urgent  response  with  the  person 
responsible  for  the  building  or  facility  it  affects.  This  score  receives  a  value  of  1.000. 

The  least  preferred  score  is  for  a  contractor  that  was  unable  to  coordinate  urgent 
responses  with  the  person  responsible  for  the  building  or  facility  it  affects.  That  score 
receives  a  value  of  0.000.  As  illustrate  in  Figure  72,  the  contractor’s  value  on  the  y-axis 
increases  for  every  increase  in  percentage  on  the  x-axis. 


Figure  72.  Percentage  of  Goal  Met  for  Effective  FACMAN  Urgent  Coordination  SDVF 
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SDVF  #  44 — Percentage  of  Goal  Met  for  Timely  Initial  Routine  Response 

The  SDVF  for  Percentage  of  Goal  Met  for  Timely  Initial  Routine  Response  is  a 
monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  initial  routine  response  goal.  The  most  preferred  score  is  for  a  contractor  that 
was  able  to  have  a  representative  on  site  to  respond  to  every  initial  routine  response  in  a 
timely  manner.  This  score  receives  a  value  of  1.000.  The  least  preferred  score  is  for  a 
contractor  that  was  unable  to  have  a  representative  on  site  to  respond  to  initial  routine 
responses  in  a  timely  manner.  That  score  receives  a  value  of  0.000.  As  illustrate  in 
Figure  73,  the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in  percentage 
on  the  x-axis. 


Figure  73.  Percentage  of  Goal  Met  for  Timely  Initial  Routine  Response  SDVF 
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SDVF  #  45 — Percentage  of  Goal  Met  for  Timely  Remedied  Routine  Response 

The  SDVF  for  Percentage  of  Goal  Met  for  Timely  Remedied  Routine  Response  is  a 
monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  remedied  routine  response  goal.  The  most  preferred  score  is  for  a  contractor 
whose  crews  were  able  to  remedy  all  routine  responses  in  a  timely  manner.  This  score 
receives  a  value  of  1 .000.  The  least  preferred  score  is  for  a  contractor  whose  crews  were 
unable  to  remedy  routine  responses  in  a  timely  manner.  That  score  receives  a  value  of 
0.000.  As  illustrate  in  Figure  74,  the  contractor’s  value  on  the  y-axis  exponentially 
increases  for  every  increase  in  percentage  on  the  x-axis. 


Figure  74.  Percentage  of  Goal  Met  for  Timely  Remedied  Routine  Response  SDVF 
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SDVF  #  46 — Percentage  of  Goal  Met  for  Effective  2-Week  Coordination 

The  SDVF  for  Percentage  of  Goal  Met  for  Effective  2 -Week  Coordination  is  a 
monotonically  increasing  function  that  measures  the  percentage  of  time  the  contractor 
met  the  two-week  coordination  goal.  The  most  preferred  score  is  for  a  contractor  to 
coordinate  with  a  contracting  office  representative  at  least  two  weeks  in  advance  for 
every  routine  response.  This  score  receives  a  value  of  1.000.  The  least  preferred  score  is 
for  a  contractor  to  not  coordinate  with  a  contracting  office  representative  at  least  two 
weeks  in  advance  for  any  routine  response.  That  score  receives  a  value  of  0.000.  As 
illustrate  in  Figure  75,  the  contractor’s  value  on  the  y-axis  increases  for  every  increase  in 
percentage  on  the  x-axis. 


Figure  75.  Percentage  of  Goal  Met  for  Effective  2-Week  Coordination  SDVF 
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SDVF  #  47 — Percentage  of  Goal  Met  for  Effective  FACMAN  Routine  Coordination 


The  SDVF  for  Percentage  of  Goal  Met  for  Effective  FACMAN  Routine 
Coordination  is  a  monotonically  increasing  function  that  measures  the  percentage  of  time 
the  contractor  met  the  FACMAN  routine  coordination  goal.  The  most  preferred  score  is 
for  a  contractor  that  was  able  to  coordinate  every  routine  response  with  the  person 
responsible  for  the  building  or  facility  it  affects.  This  score  receives  a  value  of  1.000. 

The  least  preferred  score  is  for  a  contractor  that  was  unable  to  coordinate  routine 
responses  with  the  person  responsible  for  the  building  or  facility  it  affects.  That  score 
receives  a  value  of  0.000.  As  illustrate  in  Figure  76,  the  contractor’s  value  on  the  y-axis 
increases  for  every  increase  in  percentage  on  the  x-axis. 


Figure  76.  Percentage  of  Goal  Met  for  Effective  FACMAN  Routine  Coordination  SDVF 
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Appendix  D.  Global  Weights  for  Utility  Privatization  Evaluation  Hierarchy 


lst-Tier 

Values 

2nd-Tier 

Values 

3ld-Tier  Values 
or  Measures 

4th-Tier  Values  or 
Measures 

5th-Tier  Values  or 
Measures 

Global 

Weights 

Measure 

# 

Quality 

Effective 

Administration 

Maintaining 
Proper  Licenses, 
Pennits,  & 
Certification 

%  of  Up-to-Date 
Licenses,  Permits,  and 
Certifications 

0.016 

1 

Maintaining 
Service  Records 
for  2  Years 

Are  Records  for  the 
Past  2  Years 
Maintained  Properly? 

0.014 

2 

Maintaining  and 
Updating 
Drawings 

Average  #  of  Days  to 
Update  Drawings 

0.016 

3 

Environmental 

Compliance 

Effective  Spill 
Contingency 
Plan 

#  of  Positive  Findings 
for  Spill  Contingency 
Plan 

0.005 

4 

#  of  Minor  Findings 
for  Spill  Contingency 
Plan 

0.005 

5 

#  of  Major  Findings 
for  Spill  Contingency 
Plan 

0.005 

6 

#  of  Significant 
Findings  for  Spill 
Contingency  Plan 

0.005 

7 

SO 


lst-Tier 

Values 

2nd-Tier 

Values 

3rd-Tier  Values 
or  Measures 

Quality 

Environmental 

Compliance 

Hazmat/Hazwaste 
Minimization  and 
Recycling 

Safety 


Compliance  with 
Utility  System 
Laws/Regulations 


4th-Tier  Values  or 
_ Measures _ 

%  of  Liquid  Waste 
Diverted  from 
_ Landfill _ 

%  of  Solid  Waste 
Diverted  from 
_ Landfill _ 

#  of  Positive  Findings 
for  Recycling 
_ Program _ 

#  of  Minor  Findings 

for  Recycling 
Program 

#  of  Major  Findings 

for  Recycling 
_ Program _ 

#  of  Significant 
Findings  for 
Recycling  Program 

#  of  Lost  Man-Hours 
Due  to  Utility  System 

Mishaps 


5th-Tier  Values  or 
Measures 


lst-Tier 

Values 


2nd-Tier 

Values 


3  -Tier  Values 
or  Measures 


4th-Tier  Values  or 

5th-Tier  Values  or 

Measures 

Measures 

#  of  Utility  System 

Mishaps 

#  of  RAC  1  - 

Catastrophic 

Violations 

#  of  RAC  2  -  Critical 

Violation 

#  of  RAC  3  - 

Moderate  Violations 

#  of  RAC  4  - 

Negligible  Violations 

%  of  Employees 

Completing  all 

Requirements 

lst-Tier 

Values 

2nd-Tier  Values 

3ld-Tier 
Values  or 
Measures 

Quality 

Utility  System  Security 

#  of  Employees 
Identified  as 
Potential 

Threats 

Are  all 
Employee 
Security 
Clearances  Up- 
to-Date? 

Completed 

Renewals/Replacements 

%  of  Items 
Actually 
Replaced 

Reliability 

Decreased  Utility 
System  Outages 

%  of  Critical 
Utility  System 
Outages  caused 
by  System 
Management 

%  of  Non- 
Critical  Utility 
System 

Outages  caused 
by  System 
Management 

Effective  Digging 
Permits/Line  Marking 
Program 

#  of  Utility 
Line  Hits 

rier  Values  or  5th-Tier  Values  or  Global 
Measures  Measures  Weights 


0.053 

23 

0.053 

24 

0.140 

25 

0.105 

26 

0.105 

27 

0.035 

28 

Measure 

# 


G\ 

K> 


lst-Tier 

Values 

2nd-Tier 

Values 

3rd-Tier  Values 
or  Measures 

4th-Tier  Values  or 
Measures 

5th-Tier  Values  or 
Measures 

Global 

Weights 

Measure 

# 

Response 

High 

Contracting 

Meeting 

Attendance 

%  of  Meetings 
Attended 

0.035 

29 

Timely  Meter 
Reading 

%  of  Time  Meter 
Readings  were 
Late 

0.035 

30 

Timely  Service 
Response 

Timely 

Emergency 

Service 

Response 

Adequate  24/7 
Hotline 

Is  There  an  Adequate 
24/7  Hotline? 

0.012 

31 

Timely  Initial 
Emergency  Response 

%  of  Goal  Met  for 
Timely  Initial 
Emergency  Response 

0.025 

32 

Timely  Emergency 
Crew  Response 

%  of  Goal  Met  for 
Timely  Emergency 
Crew  Response 

0.025 

33 

Timely  Remedied 
Emergency  Response 

%  of  Goal  Met  for 
Timely  Remedied 
Emergency  Response 

0.025 

34 

lst-Tier 

Values 

2nd-Tier 

Values 

3rd-Tier 
Values  or 
Measures 

4th-Jier  Values  or 
Measures 

5th-Tier  Values  or 
Measures 

Global 

Weights 

Measure 

# 

#  of  Outstanding 
Ratings  from 
Exercises/Contingencies 

0.005 

35 

#  of  Excellent 
Ratings  from 
Exercises/Contingencies 

0.005 

36 

Response 

Timely 

Service 

Response 

Timely 

Emergency 

Service 

Response 

Timely  Response  to 
Exercises/Contingencies 

#  of  Satisfactory 
Ratings  from 
Exercises/Contingencies 

0.005 

37 

#  of  Marginal 
Ratings  from 
Exercises/Contingencies 

0.005 

38 

#  of  Unsatisfactory 
Ratings  from 
Exercises/Contingencies 

0.005 

39 

lst-Tier 

Values 

2nd-Tier 

Values 

3rd-Tier  Values 
or  Measures 

Timely 

Emergency 

Service 

Response 

Response 

Timely  Service 
Response 

Timely  Urgent 
Service 
Response 

Timely  Routine 
Service 
Response 

4  -Tier  Values  or 
Measures 


Effective  FACMAN 
Emergency 
Coordination 

Timely  Initial 
Urgent  Response 

Timely  Remedied 
Urgent  Response 

Effective  FACMAN 
Urgent  Coordination 

Timely  Initial 
Routine  Response 

Timely  Remedied 
Routine  Response 


5  -Tier  Values  or 
Measures 


%  of  Goal  Met  for 
Effective  FACMAN 
Emergency 
Coordination 

%  of  Goal  Met  for 
Timely  Initial 
Urgent  Response 

%  of  Goal  Met  for 
Timely  Remedied 
Urgent  Response 

%  of  Goal  Met  for 
Effective  FACMAN 
Urgent  Coordination 

%  of  Goal  Met  for 
Timely  Initial 
Routine  Response 

%  of  Goal  Met  for 
Timely  Remedied 
Routine  Response 


Measure 

# 


0.012 


0.029 


0.029 


0.015 


0.015 


0.015 


os 


lst-Tier 

Values 

2nd-Tier 

Values 

3rd-Tier  Values 
or  Measures 

4th-Tier  Values  or 
Measures 

5th-Tier  Values  or 
Measures 

Global 

Weights 

Measure 

# 

Response 

Timely  Service 
Response 

Timely  Routine 
Service 
Response 

Effective  2- Week 
Coordination 

%  of  Goal  Met  for 
Effective  2- Week 
Coordination 

0.010 

46 

Effective  FACMAN 
Routine  Coordination 

%  of  Goal  Met  for 
Effective  FACMAN 
Routine  Coordination 

0.010 

47 

Appendix  E.  Ranked  Ordered  Global  Weights 
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Measure 

Global 

Weight 

%  of  Solid  Waste  Diverted  from  Landfills 

0.008 

#  of  RAC  2  —  Critical  Violations 

0.006 

#  of  Outstanding  Ratings  from  Exercises/Contingencies 

0.005 

#  of  Excellent  Ratings  from  Exercise/Contingencies 

0.005 

#  of  Satisfactory  Ratings  from  Exercises/Contingencies 

0.005 

#  of  Marginal  Ratings  from  Exercises/Contingencies 

0.005 

#  of  Unsatisfactory  Ratings  from  Exercises/Contingencies 

0.005 

#  of  Positive  Findings  from  Spill  Contingency  Plan 

0.005 

#  of  Significant  Findings  from  Spill  Contingency  Plan 

0.005 

#  of  Minor  Findings  from  Spill  Contingency  Plan 

0.005 

#  of  Major  Findings  from  Spill  Contingency  Plan 

0.005 

#  of  RAC  3  —  Moderate  Violations 

0.003 

#  of  Positive  Findings  for  Recycling  Program 

0.003 

#  of  Significant  Findings  for  Recycling  Program 

0.003 

#  of  Minor  Findings  for  Recycling  Program 

0.003 

#  of  Major  Findings  for  Recycling  Program 

0.003 

#  of  RAC  4  —  Negligible  Violations 

0.001 
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Appendix  F.  Notional  Data  Scoring  Results 


Branch 

Measure  # 

Measure  Title 

Base  1 

Base  2 

Base  3 

Base  4 

Base  5 

Base  6 

Base  7 

Base  8 

Quality 

1 

%  of  Up-to-Date 
Licenses,  Permits,  and 
Certifications 

98% 

98% 

99% 

82% 

86% 

67% 

89% 

100% 

2 

Are  Records  for  the  Past 

2  Years  Maintained 
Properly? 

No 

Yes 

No 

Yes 

Yes 

No 

Yes 

Yes 

3 

Average  #  of  Days  to 
Update  Drawings 

72 

74 

76 

64 

79 

69 

57 

61 

4 

#  of  Positive  Findings 
for  Spill  Contingency 
Plan 

13 

11 

3 

6 

12 

1 

14 

13 

5 

#  of  Minor  Findings  for 
Spill  Contingency  Plan 

19 

13 

11 

16 

18 

12 

0 

13 

6 

#  of  Major  Findings  for 
Spill  Contingency  Plan 

14 

2 

12 

19 

0 

16 

0 

0 

7 

#  of  Significant 
Findings  for  Spill 
Contingency  Plan 

14 

3 

12 

9 

11 

18 

0 

8 

8 

%  of  Liquid  Waste 
Diverted  from  Landfill 

93% 

100% 

79% 

93% 

83% 

84% 

100% 

91% 

G\ 

vo 


Branch 

Measure  # 

Measure  Title 

Base  1 

9 

%  of  Solid  Waste 
Diverted  from  Landfill 

63% 

10 

#  of  Positive  Findings 
for  Recycling  Program 

14 

11 

#  of  Minor  Findings  for 
Recycling  Program 

14 

12 

#  of  Major  Findings  for 
Recycling  Program 

8 

13 

#  of  Significant 
Findings  for  Recycling 
Program 

7 

Quality 

14 

#  of  Lost  Man-Hours 
Due  to  Utility  System 
Mishaps 

46 

15 

#  of  Utility  System 
Mishaps 

8 

16 

#  of  RAC  1  - 
Catastrophic  Violations 

5 

17 

#  of  RAC  2  -  Critical 
Violation 

9 

18 

#  of  RAC  3  —  Moderate 
Violations 

8 

Base  3  Base  4  Base  5  Base  6  Base  7 


99% 

92% 

87% 

78% 

90% 

100% 

89% 

14 

2 

8 

13 

19 

20 

15 

6 

17 

2 

12 

5 

0 

11 

0 

16 

4 

17 

18 

0 

13 

19 

13 

1 

10 

10 

2 

2 

13  12  11  21  85  20  15 

1  9  3  3  7  1  12 

8  0  8  5  7  0  2 

9  4  8  4  9  0  3 

5  8  3  3  8  0  3 


Base  8 


Base  2 


Branch 

Measure  # 

Measure  Title 

Base  1 

Base  2 

Base  3 

Base  4 

Base  5 

Base  6 

Base  7 

Base  8 

Quality 

19 

#  of  RAC  4  - 
Negligible  Violations 

1 

9 

0 

3 

8 

7 

0 

7 

20 

%  of  Employees 
Completing  all 
Requirements 

77% 

73% 

71% 

80% 

68% 

64% 

89% 

100% 

21 

%  of  Meters  Calibrated 
from  Random  Sample 

63% 

97% 

79% 

81% 

82% 

97% 

65% 

0% 

22 

%  of  Total  Facilities 
Metered 

63% 

65% 

69% 

75% 

86% 

85% 

60% 

87% 

23 

#  of  Employees 
Identified  as  Potential 
Threats 

4 

1 

5 

3 

0 

1 

0 

0 

24 

Are  all  Employee 
Security  Clearances  Up- 
to-Date? 

Yes 

Yes 

No 

No 

Yes 

Yes 

Yes 

Yes 

Reliability 

25 

%  of  Items  Actually 
Replaced 

65% 

72% 

70% 

89% 

95% 

77% 

100% 

78% 

26 

%  of  Critical  Utility 
System  Outages  caused 
by  System  Management 

5% 

4% 

37% 

11% 

4% 

45% 

0% 

15% 

Branch 

Measure  # 

Measure  Title 

Base  1 

Base  2 

Base  3 

Base  4 

Base  5 

Base  6 

Base  7 

Base  8 

Reliability 

27 

%  of  Non-Critical 
Utility  System  Outages 
caused  by  System 
Management 

49% 

8% 

46% 

22% 

2% 

44% 

0% 

20% 

Responsiveness 

28 

#  of  Utility  Line  Hits 

15 

0 

11 

16 

3 

7 

0 

2 

29 

%  of  Meetings  Attended 

63% 

100% 

89% 

91% 

92% 

40% 

100% 

100% 

30 

%  of  Time  Meter 
Readings  were  Late 

6% 

97% 

81% 

77% 

97% 

12% 

0% 

100% 

31 

Is  There  an  Adequate 
24/7  Hotline? 

Yes 

Yes 

No 

Yes 

Yes 

No 

Yes 

Yes 

32 

%  of  Goal  Met  for 
Timely  Initial 
Emergency  Response 

74% 

69% 

95% 

63% 

93% 

78% 

100% 

94% 

33 

%  of  Goal  Met  for 
Timely  Emergency 
Crew  Response 

86% 

81% 

87% 

65% 

99% 

89% 

100% 

99% 

34 

%  of  Goal  Met  for 
Timely  Remedied 
Emergency  Response 

96% 

96% 

85% 

82% 

100% 

67% 

100% 

100% 

35 

#  of  Outstanding 
Ratings  from 
Exercises/Contingencies 

20 

19 

9 

18 

5 

7 

23 

19 

Branch 

Measure  # 

Measure  Title 

Base  1 

Base  2 

Base  3 

Base  4 

Base  5 

Base  6 

Base  7 

Base  8 

36 

#  of  Excellent 

Ratings  from 
Exercises/Contingencies 

2 

19 

10 

19 

5 

14 

20 

20 

37 

#  of  Satisfactory 
Ratings  from 
Exercises/Contingencies 

8 

18 

2 

14 

4 

5 

20 

15 

38 

#  of  Marginal 

Ratings  from 
Exercises/Contingencies 

13 

9 

5 

15 

3 

11 

0 

2 

Responsiveness 

39 

#  of  Unsatisfactory 
Ratings  from 
Exercises/Contingencies 

8 

18 

2 

14 

0 

5 

0 

0 

40 

%  of  Goal  Met  for 
Effective  FACMAN 
Emergency 
Coordination 

80% 

93% 

79% 

83% 

100% 

66% 

100% 

100% 

41 

%  of  Goal  Met  for 
Timely  Initial  Urgent 
Response 

73% 

91% 

100% 

77% 

92% 

61% 

100% 

94% 

Branch 

Measure  # 

Measure  Title 

44 

%  of  Goal  Met  for 
Timely  Initial  Routine 
Response 

42 

%  of  Goal  Met  for 
Timely  Remedied 
Urgent  Response 

43 

%  of  Goal  Met  for 
Effective  FACMAN 
Urgent  Coordination 

Responsiveness 

45 

%  of  Goal  Met  for 
Timely  Remedied 
Routine  Response 

46 

%  of  Goal  Met  for 
Effective  2-Week 
Coordination 

47 

%  of  Goal  Met  for 
Effective  FACMAN 
Routine  Coordination 

Base  1 

Base  2 

Base  3 

Base  4 

Base  5 

Base  6 

Base  7 

Base  8 

67% 

96% 

81% 

86% 

95% 

54% 

100% 

96% 

65% 

94% 

96% 

72% 

94% 

51% 

100% 

95% 

76% 

91% 

100% 

77% 

92% 

61% 

100% 

94% 

65% 

95% 

79% 

73% 

94% 

70% 

100% 

96% 

85% 

100% 

71% 

97% 

91% 

70% 

100% 

94% 

61% 

90% 

96% 

64% 

100% 

52% 

100% 

100% 
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